2024-09-21T01:53:34.807 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T01:53:34.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T01:53:34.930 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913405' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 3 session_timeout: 300 cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: client_shutdown_timeout: 120 global: ms die on skipped message: false mds: mds oft prefetch dirfrags: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - overall HEALTH_ - \(MON_DOWN\) sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFkwIwnUexxNThooSyNon7VqKJErVIzAfege5i1SFcBthAyQJLVHRxzQ1NgNRURc1zxFWNWeJDtrugDkrC95/s= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZcvmmzjbLNJIaQiWTi2nXoZzpBq1KIOhRoAimkx9kJ5qaH+I3I8bkZbnMsS+cYL+mF4Lb8lpwmv7jt3T6e97U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T01:53:34.930 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T01:53:34.931 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T01:53:34.931 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T01:53:34.932 INFO:teuthology.task.internal:Checking packages... 2024-09-21T01:53:34.951 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T01:53:34.951 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T01:53:34.951 INFO:teuthology.packaging:ref: None 2024-09-21T01:53:34.951 INFO:teuthology.packaging:tag: None 2024-09-21T01:53:34.951 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:53:34.951 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:53:34.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:53:35.089 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T01:53:35.089 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T01:53:35.095 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T01:53:35.096 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T01:53:35.101 INFO:teuthology.task.internal:Saving configuration 2024-09-21T01:53:35.121 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T01:53:35.126 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T01:53:35.151 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:48:13.151086', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFkwIwnUexxNThooSyNon7VqKJErVIzAfege5i1SFcBthAyQJLVHRxzQ1NgNRURc1zxFWNWeJDtrugDkrC95/s='} 2024-09-21T01:53:35.175 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:48:13.152486', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZcvmmzjbLNJIaQiWTi2nXoZzpBq1KIOhRoAimkx9kJ5qaH+I3I8bkZbnMsS+cYL+mF4Lb8lpwmv7jt3T6e97U='} 2024-09-21T01:53:35.175 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T01:53:35.181 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-21T01:53:35.181 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-21T01:53:35.181 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T01:53:35.238 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3cf2510160>, signals=[15]) 2024-09-21T01:53:35.238 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T01:53:35.244 INFO:teuthology.task.internal:Opening connections... 2024-09-21T01:53:35.244 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T01:53:35.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:53:35.320 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T01:53:35.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:53:35.393 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T01:53:35.410 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-09-21T01:53:35.428 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-09-21T01:53:35.428 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-09-21T01:53:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-09-21T01:53:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-09-21T01:53:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-09-21T01:53:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:53:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-09-21T01:53:35.485 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:53:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:53:35.487 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-09-21T01:53:35.510 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-09-21T01:53:35.528 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-09-21T01:53:35.528 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-09-21T01:53:35.585 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-09-21T01:53:35.585 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-09-21T01:53:35.585 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-09-21T01:53:35.585 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-09-21T01:53:35.585 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-09-21T01:53:35.585 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T01:53:35.586 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T01:53:35.586 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-09-21T01:53:35.609 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T01:53:35.618 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T01:53:35.623 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T01:53:35.623 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:53:35.626 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:53:35.642 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T01:53:35.647 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T01:53:35.647 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:53:35.683 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:53:35.791 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T01:53:35.846 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T01:53:35.860 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T01:53:35.860 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T01:53:35.860 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T01:53:35.860 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:53:35.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:53:35.878 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-09-21T01:53:35.936 INFO:teuthology.orchestra.run.smithi088.stdout:5.14.0-503.el9.x86_64 2024-09-21T01:53:35.937 INFO:teuthology.task.kernel:Running kernel on smithi088: 5.14.0-503.el9.x86_64 2024-09-21T01:53:35.937 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum install -y kernel 2024-09-21T01:53:38.318 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-21T01:53:40.496 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-21T01:53:44.769 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-21T01:53:46.245 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 19 kB 00:00 2024-09-21T01:53:47.032 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-21T01:53:52.396 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-21T01:53:54.311 INFO:teuthology.orchestra.run.smithi088.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T01:53:54.311 INFO:teuthology.orchestra.run.smithi088.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T01:53:54.410 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-09-21T01:53:54.422 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-09-21T01:53:54.423 INFO:teuthology.orchestra.run.smithi088.stdout:Install 4 Packages 2024-09-21T01:53:54.423 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T01:53:54.423 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 79 M 2024-09-21T01:53:54.423 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 126 M 2024-09-21T01:53:54.423 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-09-21T01:53:54.556 INFO:teuthology.orchestra.run.smithi088.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 324 kB/s | 36 kB 00:00 2024-09-21T01:53:57.072 INFO:teuthology.orchestra.run.smithi088.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 6.0 MB/s | 16 MB 00:02 2024-09-21T01:53:58.847 INFO:teuthology.orchestra.run.smithi088.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 7.8 MB/s | 35 MB 00:04 2024-09-21T01:53:59.139 INFO:teuthology.orchestra.run.smithi088.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 6.2 MB/s | 29 MB 00:04 2024-09-21T01:53:59.139 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:53:59.139 INFO:teuthology.orchestra.run.smithi088.stdout:Total 17 MB/s | 79 MB 00:04 2024-09-21T01:53:59.823 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-09-21T01:54:00.190 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-09-21T01:54:00.191 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-09-21T01:54:01.161 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-09-21T01:54:01.161 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-09-21T01:54:03.581 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-09-21T01:54:04.508 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T01:54:04.591 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:54:07.340 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:54:07.468 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:54:12.282 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:54:12.346 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:54:17.186 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:54:55.284 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:54:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:54:55.623 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:54:55.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T01:54:55.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T01:54:55.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T01:54:56.500 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-09-21T01:54:56.847 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T01:54:56.928 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T01:54:56.929 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T01:54:56.929 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T01:54:56.929 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T01:54:56.929 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T01:54:56.929 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T01:54:56.929 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T01:54:56.929 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T01:54:56.988 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T01:54:56.988 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T01:54:56.988 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -qi grub2-tools 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Name : grub2-tools 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Epoch : 1 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Version : 2.06 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Release : 82.el9 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: x86_64 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Group : Unspecified 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:Size : 8274670 2024-09-21T01:54:57.020 INFO:teuthology.orchestra.run.smithi088.stdout:License : GPLv3+ 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Packager : builder@centos.org 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Vendor : CentOS 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Summary : Support tools for GRUB. 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:Description : 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout:hardware devices. 2024-09-21T01:54:57.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T01:54:57.022 INFO:teuthology.orchestra.run.smithi088.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T01:54:57.022 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T01:54:57.022 INFO:teuthology.task.kernel:Updating grub on smithi088 to boot 5.14.0-508.el9.x86_64 2024-09-21T01:54:57.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T01:54:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-09-21T01:54:59.022 INFO:teuthology.orchestra.run.smithi088.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T01:54:59.053 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-09-21T01:54:59.056 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-09-21T01:54:59.072 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.VrRafN4ogV 2024-09-21T01:54:59.072 DEBUG:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VrRafN4ogV 2024-09-21T01:54:59.138 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.VrRafN4ogV 2024-09-21T01:54:59.273 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.VrRafN4ogV is 6KB 2024-09-21T01:54:59.293 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.VrRafN4ogV 2024-09-21T01:54:59.307 DEBUG:teuthology.orchestra.run.smithi088:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T01:54:59.376 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T01:54:59.376 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T01:54:59.376 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T01:54:59.376 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T01:54:59.378 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T01:54:59.535 DEBUG:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2024-09-21T01:55:29.537 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T01:55:29.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T01:55:29.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:56:04.207 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T01:56:13.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T01:56:13.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:56:16.271 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T01:56:28.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T01:56:28.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:57:00.116 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-09-21T01:57:00.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T01:57:00.468 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T01:57:00.468 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-09-21T01:57:00.482 INFO:teuthology.orchestra.run.smithi088.stdout:5.14.0-508.el9.x86_64 2024-09-21T01:57:00.482 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T01:57:00.482 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T01:57:00.482 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T01:57:01.483 DEBUG:teuthology.parallel:result is None 2024-09-21T01:57:01.483 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T01:57:01.489 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T01:57:01.489 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:57:01.492 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:57:01.511 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T01:57:01.519 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T01:57:01.527 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T01:57:01.527 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:57:01.557 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:57:01.618 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T01:57:01.626 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T01:57:01.626 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:57:01.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:57:01.643 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:57:01.645 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:57:01.724 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:01.730 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:01.743 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:01.983 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:57:01.986 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T01:57:01.995 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T01:57:01.995 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:57:01.998 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:57:02.051 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T01:57:02.065 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T01:57:02.065 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:57:02.070 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:57:02.096 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:57:02.205 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:57:02.304 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:57:02.371 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:57:02.452 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T01:57:02.452 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:57:02.510 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:57:02.555 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:57:02.610 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:57:02.695 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:57:02.736 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T01:57:02.736 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:57:02.802 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-09-21T01:57:02.805 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-09-21T01:57:02.877 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:57:03.006 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:57:03.570 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T01:57:03.579 INFO:teuthology.task.internal:Starting timer... 2024-09-21T01:57:03.579 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T01:57:03.652 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T01:57:03.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2024-09-21T01:57:03.707 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2024-09-21T01:57:03.709 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-09-21T01:57:03.769 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-09-21T01:57:03.771 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T01:57:03.771 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2024-09-21T01:57:03.794 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2024-09-21T01:57:03.795 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-09-21T01:57:03.821 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-09-21T01:57:03.821 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi088.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-09-21T01:57:03.822 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T01:57:03.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:57:03.863 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T01:57:03.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:57:03.891 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T01:57:03.892 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:57:03.942 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:57:03.971 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T01:57:03.981 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T01:57:03.992 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T01:57:03.992 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi088.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T02:01:00.017 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-09-21T02:01:00.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T02:01:00.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:00.092 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-09-21T02:01:00.181 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T02:01:00.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-21T02:01:00.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:00.255 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-21T02:01:00.338 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-21T02:01:00.338 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T02:01:00.349 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T02:01:00.350 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:01:00.350 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:01:00.352 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:01:00.352 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:01:00.378 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:01:00.393 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:01:00.408 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:01:00.417 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2024-09-21T02:01:00.422 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:01:00.428 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2024-09-21T02:01:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:01:00.447 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-09-21T02:01:00.455 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:01:00.459 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-09-21T02:01:00.472 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:01:00.487 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:01:00.518 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:01:00.522 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:01:00.523 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-09-21T02:01:00.523 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.523 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.523 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.523 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.549 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:01:00.552 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:01:00.552 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-09-21T02:01:00.552 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.552 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.553 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.553 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:01:00.554 INFO:teuthology.run_tasks:Running task install... 2024-09-21T02:01:00.561 DEBUG:teuthology.task.install:project ceph 2024-09-21T02:01:00.562 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T02:01:00.562 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T02:01:00.562 INFO:teuthology.task.install:Using flavor: default 2024-09-21T02:01:00.567 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T02:01:00.568 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T02:01:00.568 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:01:00.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:01:00.570 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:01:00.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:01:00.719 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:01:00.720 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:01:00.723 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:01:00.723 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:01:00.829 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:01:00.829 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:01:00.829 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:01:00.834 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:01:00.834 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:01:00.834 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:01:00.854 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:01:00.855 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:01:00.859 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:01:00.859 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:01:00.920 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:01:00.925 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:01:00.994 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:01:01.000 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:01:01.059 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2024-09-21T02:01:01.060 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2024-09-21T02:01:01.066 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-09-21T02:01:01.066 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-09-21T02:01:01.291 INFO:teuthology.orchestra.run.smithi195.stdout:45 files removed 2024-09-21T02:01:01.302 INFO:teuthology.orchestra.run.smithi088.stdout:45 files removed 2024-09-21T02:01:01.318 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:01:01.329 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:01:01.850 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 336 kB/s | 84 kB 00:00 2024-09-21T02:01:01.873 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 327 kB/s | 84 kB 00:00 2024-09-21T02:01:02.134 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T02:01:02.157 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-21T02:01:02.410 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T02:01:02.433 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T02:01:02.935 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-21T02:01:04.008 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.3 MB 00:01 2024-09-21T02:01:04.745 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-21T02:01:09.583 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-21T02:01:10.320 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T02:01:11.788 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-21T02:01:12.892 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T02:01:14.292 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.5 MB 00:00 2024-09-21T02:01:16.101 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-21T02:01:17.205 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T02:01:18.564 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 4.8 kB/s | 1.7 kB 00:00 2024-09-21T02:01:20.708 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:01:20.709 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:01:20.720 INFO:teuthology.orchestra.run.smithi195.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:01:20.721 INFO:teuthology.orchestra.run.smithi195.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:01:20.722 INFO:teuthology.orchestra.run.smithi195.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:01:20.723 INFO:teuthology.orchestra.run.smithi195.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:01:20.725 INFO:teuthology.orchestra.run.smithi195.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:01:20.727 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:01:20.728 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:01:20.791 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:01:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:01:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:01:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:01:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:01:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:01:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:01:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:01:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:01:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:01:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:01:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:01:20.820 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout:Install 149 Packages 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 8 Packages 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 228 M 2024-09-21T02:01:20.821 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-09-21T02:01:21.724 INFO:teuthology.orchestra.run.smithi195.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-21T02:01:21.916 INFO:teuthology.orchestra.run.smithi195.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.4 MB/s | 873 kB 00:00 2024-09-21T02:01:21.984 INFO:teuthology.orchestra.run.smithi195.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T02:01:22.067 INFO:teuthology.orchestra.run.smithi195.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-21T02:01:22.192 INFO:teuthology.orchestra.run.smithi195.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 11 MB/s | 2.3 MB 00:00 2024-09-21T02:01:22.234 INFO:teuthology.orchestra.run.smithi195.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-09-21T02:01:22.593 INFO:teuthology.orchestra.run.smithi195.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.0 MB 00:00 2024-09-21T02:01:22.735 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T02:01:22.785 INFO:teuthology.orchestra.run.smithi195.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T02:01:22.826 INFO:teuthology.orchestra.run.smithi195.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 607 kB/s | 25 kB 00:00 2024-09-21T02:01:23.327 INFO:teuthology.orchestra.run.smithi195.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 16 MB/s | 18 MB 00:01 2024-09-21T02:01:23.435 INFO:teuthology.orchestra.run.smithi195.stdout:(11/157): ceph-radosgw-19.3.0-5120.g18432218.el 12 MB/s | 10 MB 00:00 2024-09-21T02:01:23.461 INFO:teuthology.orchestra.run.smithi195.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.7 MB/s | 226 kB 00:00 2024-09-21T02:01:23.486 INFO:teuthology.orchestra.run.smithi195.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 538 kB/s | 27 kB 00:00 2024-09-21T02:01:23.511 INFO:teuthology.orchestra.run.smithi195.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 631 kB/s | 32 kB 00:00 2024-09-21T02:01:23.562 INFO:teuthology.orchestra.run.smithi195.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T02:01:23.587 INFO:teuthology.orchestra.run.smithi195.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T02:01:23.620 INFO:teuthology.orchestra.run.smithi195.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T02:01:23.662 INFO:teuthology.orchestra.run.smithi195.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T02:01:23.721 INFO:teuthology.orchestra.run.smithi195.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 773 kB/s | 45 kB 00:00 2024-09-21T02:01:23.788 INFO:teuthology.orchestra.run.smithi195.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.2 MB/s | 148 kB 00:00 2024-09-21T02:01:23.855 INFO:teuthology.orchestra.run.smithi195.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.4 MB/s | 161 kB 00:00 2024-09-21T02:01:23.922 INFO:teuthology.orchestra.run.smithi195.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.7 MB/s | 321 kB 00:00 2024-09-21T02:01:23.989 INFO:teuthology.orchestra.run.smithi195.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T02:01:24.106 INFO:teuthology.orchestra.run.smithi195.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.3 MB 00:00 2024-09-21T02:01:24.131 INFO:teuthology.orchestra.run.smithi195.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 701 kB/s | 99 kB 00:00 2024-09-21T02:01:24.165 INFO:teuthology.orchestra.run.smithi195.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T02:01:24.232 INFO:teuthology.orchestra.run.smithi195.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T02:01:24.324 INFO:teuthology.orchestra.run.smithi195.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 16 MB/s | 3.1 MB 00:00 2024-09-21T02:01:24.349 INFO:teuthology.orchestra.run.smithi195.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 292 kB/s | 34 kB 00:00 2024-09-21T02:01:24.376 INFO:teuthology.orchestra.run.smithi195.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.9 MB/s | 154 kB 00:00 2024-09-21T02:01:24.851 INFO:teuthology.orchestra.run.smithi195.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 20 MB/s | 40 MB 00:02 2024-09-21T02:01:24.910 INFO:teuthology.orchestra.run.smithi195.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T02:01:24.921 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:01:24.921 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:01:24.933 INFO:teuthology.orchestra.run.smithi088.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:01:24.934 INFO:teuthology.orchestra.run.smithi088.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:01:24.935 INFO:teuthology.orchestra.run.smithi088.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:01:24.936 INFO:teuthology.orchestra.run.smithi088.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:01:24.937 INFO:teuthology.orchestra.run.smithi088.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:01:24.940 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:01:24.941 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:01:24.994 INFO:teuthology.orchestra.run.smithi195.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T02:01:25.002 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-09-21T02:01:25.019 INFO:teuthology.orchestra.run.smithi195.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 455 kB/s | 49 kB 00:00 2024-09-21T02:01:25.023 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-09-21T02:01:25.023 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-09-21T02:01:25.023 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-09-21T02:01:25.023 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-09-21T02:01:25.023 INFO:teuthology.orchestra.run.smithi088.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:01:25.023 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:01:25.023 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:01:25.024 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:01:25.025 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:01:25.026 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:01:25.027 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:01:25.028 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:01:25.029 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:01:25.030 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:01:25.031 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:01:25.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:01:25.033 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:01:25.034 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout:Installing weak dependencies: 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout:Install 149 Packages 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 8 Packages 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 228 M 2024-09-21T02:01:25.035 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-09-21T02:01:25.044 INFO:teuthology.orchestra.run.smithi195.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T02:01:25.086 INFO:teuthology.orchestra.run.smithi195.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.2 MB/s | 285 kB 00:00 2024-09-21T02:01:25.128 INFO:teuthology.orchestra.run.smithi195.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 9.2 MB/s | 784 kB 00:00 2024-09-21T02:01:25.154 INFO:teuthology.orchestra.run.smithi195.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 275 kB/s | 18 kB 00:00 2024-09-21T02:01:25.295 INFO:teuthology.orchestra.run.smithi195.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 479 kB/s | 80 kB 00:00 2024-09-21T02:01:25.479 INFO:teuthology.orchestra.run.smithi195.stdout:(40/157): ceph-mgr-dashboard-19.3.0-5120.g18432 13 MB/s | 14 MB 00:01 2024-09-21T02:01:25.513 INFO:teuthology.orchestra.run.smithi195.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 882 kB/s | 191 kB 00:00 2024-09-21T02:01:25.538 INFO:teuthology.orchestra.run.smithi195.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 801 kB 00:00 2024-09-21T02:01:25.588 INFO:teuthology.orchestra.run.smithi195.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T02:01:25.647 INFO:teuthology.orchestra.run.smithi195.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T02:01:25.672 INFO:teuthology.orchestra.run.smithi195.stdout:(45/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-21T02:01:25.697 INFO:teuthology.orchestra.run.smithi195.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T02:01:25.756 INFO:teuthology.orchestra.run.smithi195.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T02:01:25.786 INFO:teuthology.orchestra.run.smithi088.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 55 kB/s | 6.6 kB 00:00 2024-09-21T02:01:25.815 INFO:teuthology.orchestra.run.smithi195.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-09-21T02:01:25.873 INFO:teuthology.orchestra.run.smithi195.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-21T02:01:25.915 INFO:teuthology.orchestra.run.smithi195.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 521 kB/s | 22 kB 00:00 2024-09-21T02:01:25.940 INFO:teuthology.orchestra.run.smithi195.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 852 kB/s | 107 kB 00:00 2024-09-21T02:01:25.983 INFO:teuthology.orchestra.run.smithi195.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-21T02:01:26.008 INFO:teuthology.orchestra.run.smithi195.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-21T02:01:26.020 INFO:teuthology.orchestra.run.smithi088.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.7 MB/s | 873 kB 00:00 2024-09-21T02:01:26.041 INFO:teuthology.orchestra.run.smithi195.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-21T02:01:26.075 INFO:teuthology.orchestra.run.smithi195.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-21T02:01:26.087 INFO:teuthology.orchestra.run.smithi088.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T02:01:26.092 INFO:teuthology.orchestra.run.smithi195.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 220 kB/s | 11 kB 00:00 2024-09-21T02:01:26.142 INFO:teuthology.orchestra.run.smithi195.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-21T02:01:26.168 INFO:teuthology.orchestra.run.smithi195.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-21T02:01:26.170 INFO:teuthology.orchestra.run.smithi088.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T02:01:26.184 INFO:teuthology.orchestra.run.smithi195.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T02:01:26.243 INFO:teuthology.orchestra.run.smithi195.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-21T02:01:26.277 INFO:teuthology.orchestra.run.smithi195.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 737 kB/s | 24 kB 00:00 2024-09-21T02:01:26.295 INFO:teuthology.orchestra.run.smithi088.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 12 MB/s | 1.5 MB 00:00 2024-09-21T02:01:26.310 INFO:teuthology.orchestra.run.smithi195.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T02:01:26.369 INFO:teuthology.orchestra.run.smithi195.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-21T02:01:26.371 INFO:teuthology.orchestra.run.smithi088.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 8.1 MB/s | 2.3 MB 00:00 2024-09-21T02:01:26.428 INFO:teuthology.orchestra.run.smithi195.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T02:01:26.453 INFO:teuthology.orchestra.run.smithi195.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T02:01:26.478 INFO:teuthology.orchestra.run.smithi195.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-21T02:01:26.605 INFO:teuthology.orchestra.run.smithi088.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 16 MB/s | 5.0 MB 00:00 2024-09-21T02:01:26.789 INFO:teuthology.orchestra.run.smithi088.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T02:01:26.847 INFO:teuthology.orchestra.run.smithi088.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 434 kB/s | 25 kB 00:00 2024-09-21T02:01:26.853 INFO:teuthology.orchestra.run.smithi195.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.4 MB/s | 3.0 MB 00:00 2024-09-21T02:01:27.296 INFO:teuthology.orchestra.run.smithi195.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-09-21T02:01:27.554 INFO:teuthology.orchestra.run.smithi195.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.4 MB/s | 910 kB 00:00 2024-09-21T02:01:27.572 INFO:teuthology.orchestra.run.smithi088.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 11 MB/s | 10 MB 00:00 2024-09-21T02:01:27.621 INFO:teuthology.orchestra.run.smithi195.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T02:01:27.738 INFO:teuthology.orchestra.run.smithi195.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.8 MB/s | 4.9 MB 00:01 2024-09-21T02:01:27.747 INFO:teuthology.orchestra.run.smithi088.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 13 MB/s | 18 MB 00:01 2024-09-21T02:01:27.781 INFO:teuthology.orchestra.run.smithi088.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.1 MB/s | 226 kB 00:00 2024-09-21T02:01:27.806 INFO:teuthology.orchestra.run.smithi088.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 464 kB/s | 27 kB 00:00 2024-09-21T02:01:27.813 INFO:teuthology.orchestra.run.smithi195.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-09-21T02:01:27.831 INFO:teuthology.orchestra.run.smithi088.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 632 kB/s | 32 kB 00:00 2024-09-21T02:01:27.871 INFO:teuthology.orchestra.run.smithi195.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T02:01:27.873 INFO:teuthology.orchestra.run.smithi088.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T02:01:27.898 INFO:teuthology.orchestra.run.smithi088.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T02:01:27.924 INFO:teuthology.orchestra.run.smithi088.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T02:01:27.949 INFO:teuthology.orchestra.run.smithi088.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 9.8 MB/s | 507 kB 00:00 2024-09-21T02:01:28.008 INFO:teuthology.orchestra.run.smithi088.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 772 kB/s | 45 kB 00:00 2024-09-21T02:01:28.097 INFO:teuthology.orchestra.run.smithi195.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-21T02:01:28.108 INFO:teuthology.orchestra.run.smithi088.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.5 MB/s | 148 kB 00:00 2024-09-21T02:01:28.139 INFO:teuthology.orchestra.run.smithi195.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T02:01:28.231 INFO:teuthology.orchestra.run.smithi195.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T02:01:28.233 INFO:teuthology.orchestra.run.smithi088.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.3 MB 00:00 2024-09-21T02:01:28.259 INFO:teuthology.orchestra.run.smithi088.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.0 MB/s | 161 kB 00:00 2024-09-21T02:01:28.284 INFO:teuthology.orchestra.run.smithi088.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T02:01:28.309 INFO:teuthology.orchestra.run.smithi088.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T02:01:28.314 INFO:teuthology.orchestra.run.smithi195.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-21T02:01:28.334 INFO:teuthology.orchestra.run.smithi088.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T02:01:28.356 INFO:teuthology.orchestra.run.smithi195.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-09-21T02:01:28.360 INFO:teuthology.orchestra.run.smithi088.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T02:01:28.398 INFO:teuthology.orchestra.run.smithi195.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 461 kB/s | 19 kB 00:00 2024-09-21T02:01:28.418 INFO:teuthology.orchestra.run.smithi088.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T02:01:28.468 INFO:teuthology.orchestra.run.smithi088.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 23 MB/s | 3.1 MB 00:00 2024-09-21T02:01:28.473 INFO:teuthology.orchestra.run.smithi195.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-21T02:01:28.494 INFO:teuthology.orchestra.run.smithi088.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 455 kB/s | 34 kB 00:00 2024-09-21T02:01:28.515 INFO:teuthology.orchestra.run.smithi195.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 835 kB/s | 35 kB 00:00 2024-09-21T02:01:28.519 INFO:teuthology.orchestra.run.smithi088.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T02:01:28.878 INFO:teuthology.orchestra.run.smithi088.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 20 MB/s | 40 MB 00:02 2024-09-21T02:01:28.936 INFO:teuthology.orchestra.run.smithi088.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T02:01:29.012 INFO:teuthology.orchestra.run.smithi088.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 660 kB/s | 49 kB 00:00 2024-09-21T02:01:29.112 INFO:teuthology.orchestra.run.smithi088.stdout:(34/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T02:01:29.137 INFO:teuthology.orchestra.run.smithi088.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 131 kB/s | 16 kB 00:00 2024-09-21T02:01:29.162 INFO:teuthology.orchestra.run.smithi088.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T02:01:29.196 INFO:teuthology.orchestra.run.smithi088.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T02:01:29.213 INFO:teuthology.orchestra.run.smithi088.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 366 kB/s | 18 kB 00:00 2024-09-21T02:01:29.357 INFO:teuthology.orchestra.run.smithi195.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-09-21T02:01:29.388 INFO:teuthology.orchestra.run.smithi088.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 16 MB/s | 14 MB 00:00 2024-09-21T02:01:29.413 INFO:teuthology.orchestra.run.smithi088.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-21T02:01:29.449 INFO:teuthology.orchestra.run.smithi195.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-21T02:01:29.533 INFO:teuthology.orchestra.run.smithi195.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T02:01:29.547 INFO:teuthology.orchestra.run.smithi088.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-21T02:01:29.572 INFO:teuthology.orchestra.run.smithi088.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-21T02:01:29.597 INFO:teuthology.orchestra.run.smithi088.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 801 kB 00:00 2024-09-21T02:01:29.622 INFO:teuthology.orchestra.run.smithi088.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T02:01:29.633 INFO:teuthology.orchestra.run.smithi195.stdout:(85/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-21T02:01:29.648 INFO:teuthology.orchestra.run.smithi088.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T02:01:29.658 INFO:teuthology.orchestra.run.smithi195.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 615 kB/s | 77 kB 00:00 2024-09-21T02:01:29.673 INFO:teuthology.orchestra.run.smithi088.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T02:01:29.684 INFO:teuthology.orchestra.run.smithi195.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T02:01:29.751 INFO:teuthology.orchestra.run.smithi195.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-21T02:01:29.776 INFO:teuthology.orchestra.run.smithi195.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-21T02:01:29.810 INFO:teuthology.orchestra.run.smithi195.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-21T02:01:29.823 INFO:teuthology.orchestra.run.smithi088.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-21T02:01:29.843 INFO:teuthology.orchestra.run.smithi195.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-21T02:01:29.866 INFO:teuthology.orchestra.run.smithi088.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 987 kB/s | 215 kB 00:00 2024-09-21T02:01:29.869 INFO:teuthology.orchestra.run.smithi195.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-21T02:01:29.911 INFO:teuthology.orchestra.run.smithi195.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-21T02:01:29.953 INFO:teuthology.orchestra.run.smithi195.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T02:01:30.036 INFO:teuthology.orchestra.run.smithi195.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-21T02:01:30.070 INFO:teuthology.orchestra.run.smithi195.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-09-21T02:01:30.153 INFO:teuthology.orchestra.run.smithi195.stdout:(97/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-21T02:01:30.170 INFO:teuthology.orchestra.run.smithi195.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-21T02:01:30.204 INFO:teuthology.orchestra.run.smithi195.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-09-21T02:01:30.263 INFO:teuthology.orchestra.run.smithi195.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T02:01:30.288 INFO:teuthology.orchestra.run.smithi195.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-21T02:01:30.321 INFO:teuthology.orchestra.run.smithi195.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T02:01:30.338 INFO:teuthology.orchestra.run.smithi195.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-21T02:01:30.408 INFO:teuthology.orchestra.run.smithi088.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 939 kB/s | 689 kB 00:00 2024-09-21T02:01:30.433 INFO:teuthology.orchestra.run.smithi088.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 189 kB/s | 107 kB 00:00 2024-09-21T02:01:30.525 INFO:teuthology.orchestra.run.smithi088.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-21T02:01:30.641 INFO:teuthology.orchestra.run.smithi088.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-09-21T02:01:30.667 INFO:teuthology.orchestra.run.smithi088.stdout:(53/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-21T02:01:30.775 INFO:teuthology.orchestra.run.smithi088.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-21T02:01:30.809 INFO:teuthology.orchestra.run.smithi088.stdout:(55/157): flex-2.6.4-9.el9.x86_64.rpm 834 kB/s | 313 kB 00:00 2024-09-21T02:01:30.876 INFO:teuthology.orchestra.run.smithi088.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-21T02:01:30.897 INFO:teuthology.orchestra.run.smithi195.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-09-21T02:01:30.918 INFO:teuthology.orchestra.run.smithi088.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-21T02:01:30.956 INFO:teuthology.orchestra.run.smithi195.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-21T02:01:30.989 INFO:teuthology.orchestra.run.smithi195.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T02:01:30.993 INFO:teuthology.orchestra.run.smithi088.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-21T02:01:31.014 INFO:teuthology.orchestra.run.smithi195.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-21T02:01:31.018 INFO:teuthology.orchestra.run.smithi088.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-09-21T02:01:31.160 INFO:teuthology.orchestra.run.smithi088.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-09-21T02:01:31.181 INFO:teuthology.orchestra.run.smithi195.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-09-21T02:01:31.207 INFO:teuthology.orchestra.run.smithi195.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T02:01:31.219 INFO:teuthology.orchestra.run.smithi088.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-21T02:01:31.240 INFO:teuthology.orchestra.run.smithi195.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 957 kB/s | 863 kB 00:00 2024-09-21T02:01:31.265 INFO:teuthology.orchestra.run.smithi195.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-09-21T02:01:31.269 INFO:teuthology.orchestra.run.smithi088.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-21T02:01:31.299 INFO:teuthology.orchestra.run.smithi195.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.7 MB/s | 151 kB 00:00 2024-09-21T02:01:31.328 INFO:teuthology.orchestra.run.smithi088.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-21T02:01:31.333 INFO:teuthology.orchestra.run.smithi195.stdout:(113/157): libunwind-1.6.2-1.el9.x86_64.rpm 727 kB/s | 67 kB 00:00 2024-09-21T02:01:31.367 INFO:teuthology.orchestra.run.smithi195.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-21T02:01:31.386 INFO:teuthology.orchestra.run.smithi088.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T02:01:31.392 INFO:teuthology.orchestra.run.smithi195.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T02:01:31.411 INFO:teuthology.orchestra.run.smithi088.stdout:(65/157): libtool-2.4.6-46.el9.x86_64.rpm 1.4 MB/s | 581 kB 00:00 2024-09-21T02:01:31.417 INFO:teuthology.orchestra.run.smithi195.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T02:01:31.443 INFO:teuthology.orchestra.run.smithi195.stdout:(117/157): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-21T02:01:31.445 INFO:teuthology.orchestra.run.smithi088.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T02:01:31.468 INFO:teuthology.orchestra.run.smithi195.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T02:01:31.493 INFO:teuthology.orchestra.run.smithi195.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T02:01:31.495 INFO:teuthology.orchestra.run.smithi088.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-21T02:01:31.527 INFO:teuthology.orchestra.run.smithi195.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T02:01:31.552 INFO:teuthology.orchestra.run.smithi195.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T02:01:31.577 INFO:teuthology.orchestra.run.smithi195.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T02:01:31.603 INFO:teuthology.orchestra.run.smithi195.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T02:01:31.628 INFO:teuthology.orchestra.run.smithi195.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T02:01:31.629 INFO:teuthology.orchestra.run.smithi088.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T02:01:31.653 INFO:teuthology.orchestra.run.smithi195.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-21T02:01:31.679 INFO:teuthology.orchestra.run.smithi195.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-21T02:01:31.704 INFO:teuthology.orchestra.run.smithi195.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-21T02:01:31.733 INFO:teuthology.orchestra.run.smithi195.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 675 kB/s | 20 kB 00:00 2024-09-21T02:01:31.772 INFO:teuthology.orchestra.run.smithi088.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 6.2 MB/s | 910 kB 00:00 2024-09-21T02:01:31.788 INFO:teuthology.orchestra.run.smithi195.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-21T02:01:31.813 INFO:teuthology.orchestra.run.smithi195.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T02:01:31.838 INFO:teuthology.orchestra.run.smithi195.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T02:01:31.864 INFO:teuthology.orchestra.run.smithi195.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T02:01:31.887 INFO:teuthology.orchestra.run.smithi088.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-21T02:01:31.906 INFO:teuthology.orchestra.run.smithi195.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.9 MB/s | 1.0 MB 00:00 2024-09-21T02:01:31.931 INFO:teuthology.orchestra.run.smithi195.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T02:01:31.948 INFO:teuthology.orchestra.run.smithi195.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 392 kB/s | 16 kB 00:00 2024-09-21T02:01:31.973 INFO:teuthology.orchestra.run.smithi195.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-21T02:01:31.990 INFO:teuthology.orchestra.run.smithi195.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-09-21T02:01:32.007 INFO:teuthology.orchestra.run.smithi195.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 915 kB/s | 31 kB 00:00 2024-09-21T02:01:32.033 INFO:teuthology.orchestra.run.smithi195.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T02:01:32.058 INFO:teuthology.orchestra.run.smithi195.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-09-21T02:01:32.083 INFO:teuthology.orchestra.run.smithi195.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 708 kB/s | 36 kB 00:00 2024-09-21T02:01:32.108 INFO:teuthology.orchestra.run.smithi195.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T02:01:32.134 INFO:teuthology.orchestra.run.smithi195.stdout:(143/157): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-21T02:01:32.159 INFO:teuthology.orchestra.run.smithi195.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T02:01:32.184 INFO:teuthology.orchestra.run.smithi195.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-21T02:01:32.196 INFO:teuthology.orchestra.run.smithi088.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 6.3 MB/s | 4.9 MB 00:00 2024-09-21T02:01:32.210 INFO:teuthology.orchestra.run.smithi195.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-21T02:01:32.235 INFO:teuthology.orchestra.run.smithi195.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-21T02:01:32.302 INFO:teuthology.orchestra.run.smithi195.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T02:01:32.355 INFO:teuthology.orchestra.run.smithi088.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-09-21T02:01:32.397 INFO:teuthology.orchestra.run.smithi088.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-21T02:01:32.494 INFO:teuthology.orchestra.run.smithi195.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 13 MB/s | 3.4 MB 00:00 2024-09-21T02:01:32.588 INFO:teuthology.orchestra.run.smithi088.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 375 kB/s | 87 kB 00:00 2024-09-21T02:01:32.619 INFO:teuthology.orchestra.run.smithi195.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 10 MB/s | 3.2 MB 00:00 2024-09-21T02:01:32.705 INFO:teuthology.orchestra.run.smithi088.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-09-21T02:01:32.936 INFO:teuthology.orchestra.run.smithi195.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 5.2 MB/s | 2.3 MB 00:00 2024-09-21T02:01:32.964 INFO:teuthology.orchestra.run.smithi088.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-21T02:01:32.978 INFO:teuthology.orchestra.run.smithi195.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 985 kB/s | 41 kB 00:00 2024-09-21T02:01:33.037 INFO:teuthology.orchestra.run.smithi195.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-21T02:01:33.071 INFO:teuthology.orchestra.run.smithi195.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 4.0 MB/s | 1.8 MB 00:00 2024-09-21T02:01:33.081 INFO:teuthology.orchestra.run.smithi088.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-21T02:01:33.140 INFO:teuthology.orchestra.run.smithi088.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 564 kB/s | 245 kB 00:00 2024-09-21T02:01:33.187 INFO:teuthology.orchestra.run.smithi195.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-21T02:01:33.190 INFO:teuthology.orchestra.run.smithi088.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 235 kB/s | 26 kB 00:00 2024-09-21T02:01:33.249 INFO:teuthology.orchestra.run.smithi088.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-21T02:01:33.307 INFO:teuthology.orchestra.run.smithi088.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-21T02:01:33.358 INFO:teuthology.orchestra.run.smithi088.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-09-21T02:01:33.546 INFO:teuthology.orchestra.run.smithi195.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.0 MB/s | 2.4 MB 00:00 2024-09-21T02:01:33.766 INFO:teuthology.orchestra.run.smithi088.stdout:(83/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-21T02:01:33.791 INFO:teuthology.orchestra.run.smithi088.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 578 kB/s | 250 kB 00:00 2024-09-21T02:01:33.992 INFO:teuthology.orchestra.run.smithi088.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 386 kB/s | 77 kB 00:00 2024-09-21T02:01:34.108 INFO:teuthology.orchestra.run.smithi088.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-09-21T02:01:34.133 INFO:teuthology.orchestra.run.smithi088.stdout:(87/157): python3-oauthlib-3.1.1-5.el9.noarch.r 605 kB/s | 222 kB 00:00 2024-09-21T02:01:34.418 INFO:teuthology.orchestra.run.smithi088.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 560 kB/s | 159 kB 00:00 2024-09-21T02:01:34.576 INFO:teuthology.orchestra.run.smithi088.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 577 kB/s | 269 kB 00:00 2024-09-21T02:01:34.693 INFO:teuthology.orchestra.run.smithi088.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-09-21T02:01:34.801 INFO:teuthology.orchestra.run.smithi088.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-09-21T02:01:34.877 INFO:teuthology.orchestra.run.smithi088.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 607 kB/s | 279 kB 00:00 2024-09-21T02:01:34.929 INFO:teuthology.orchestra.run.smithi195.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.2 MB/s | 16 MB 00:05 2024-09-21T02:01:34.936 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:01:34.936 INFO:teuthology.orchestra.run.smithi195.stdout:Total 16 MB/s | 228 MB 00:14 2024-09-21T02:01:34.985 INFO:teuthology.orchestra.run.smithi088.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-09-21T02:01:35.094 INFO:teuthology.orchestra.run.smithi088.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 409 kB/s | 44 kB 00:00 2024-09-21T02:01:35.469 INFO:teuthology.orchestra.run.smithi088.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 589 kB/s | 221 kB 00:00 2024-09-21T02:01:35.586 INFO:teuthology.orchestra.run.smithi088.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-09-21T02:01:35.695 INFO:teuthology.orchestra.run.smithi088.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-09-21T02:01:35.872 INFO:teuthology.orchestra.run.smithi088.stdout:(98/157): postgresql-server-13.16-1.el9.x86_64. 1.4 MB/s | 5.8 MB 00:03 2024-09-21T02:01:35.978 INFO:teuthology.orchestra.run.smithi088.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 545 kB/s | 58 kB 00:00 2024-09-21T02:01:36.154 INFO:teuthology.orchestra.run.smithi088.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 663 kB/s | 304 kB 00:00 2024-09-21T02:01:36.171 INFO:teuthology.orchestra.run.smithi088.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 499 kB/s | 96 kB 00:00 2024-09-21T02:01:36.183 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-09-21T02:01:36.279 INFO:teuthology.orchestra.run.smithi088.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-09-21T02:01:36.335 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-09-21T02:01:36.335 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-09-21T02:01:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-09-21T02:01:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-21T02:01:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T02:01:36.455 INFO:teuthology.orchestra.run.smithi088.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T02:01:36.472 INFO:teuthology.orchestra.run.smithi088.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-21T02:01:36.506 INFO:teuthology.orchestra.run.smithi088.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-09-21T02:01:36.556 INFO:teuthology.orchestra.run.smithi088.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-09-21T02:01:36.590 INFO:teuthology.orchestra.run.smithi088.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-21T02:01:36.623 INFO:teuthology.orchestra.run.smithi088.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T02:01:36.657 INFO:teuthology.orchestra.run.smithi088.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-21T02:01:36.724 INFO:teuthology.orchestra.run.smithi088.stdout:(113/157): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-21T02:01:36.766 INFO:teuthology.orchestra.run.smithi088.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-21T02:01:36.791 INFO:teuthology.orchestra.run.smithi088.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T02:01:36.816 INFO:teuthology.orchestra.run.smithi088.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-21T02:01:36.841 INFO:teuthology.orchestra.run.smithi088.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-21T02:01:36.867 INFO:teuthology.orchestra.run.smithi088.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T02:01:36.892 INFO:teuthology.orchestra.run.smithi088.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-21T02:01:36.917 INFO:teuthology.orchestra.run.smithi088.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T02:01:36.943 INFO:teuthology.orchestra.run.smithi088.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T02:01:36.968 INFO:teuthology.orchestra.run.smithi088.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T02:01:36.993 INFO:teuthology.orchestra.run.smithi088.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T02:01:37.018 INFO:teuthology.orchestra.run.smithi088.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-21T02:01:37.044 INFO:teuthology.orchestra.run.smithi088.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-21T02:01:37.069 INFO:teuthology.orchestra.run.smithi088.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-09-21T02:01:37.111 INFO:teuthology.orchestra.run.smithi088.stdout:(127/157): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-09-21T02:01:37.136 INFO:teuthology.orchestra.run.smithi088.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-09-21T02:01:37.161 INFO:teuthology.orchestra.run.smithi088.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T02:01:37.187 INFO:teuthology.orchestra.run.smithi088.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T02:01:37.212 INFO:teuthology.orchestra.run.smithi088.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T02:01:37.254 INFO:teuthology.orchestra.run.smithi088.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-21T02:01:37.279 INFO:teuthology.orchestra.run.smithi088.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-09-21T02:01:37.304 INFO:teuthology.orchestra.run.smithi088.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-21T02:01:37.330 INFO:teuthology.orchestra.run.smithi088.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-21T02:01:37.355 INFO:teuthology.orchestra.run.smithi088.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T02:01:37.380 INFO:teuthology.orchestra.run.smithi088.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-21T02:01:37.405 INFO:teuthology.orchestra.run.smithi088.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T02:01:37.431 INFO:teuthology.orchestra.run.smithi088.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T02:01:37.456 INFO:teuthology.orchestra.run.smithi088.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T02:01:37.481 INFO:teuthology.orchestra.run.smithi088.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T02:01:37.507 INFO:teuthology.orchestra.run.smithi088.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T02:01:37.532 INFO:teuthology.orchestra.run.smithi088.stdout:(143/157): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-21T02:01:37.557 INFO:teuthology.orchestra.run.smithi088.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T02:01:37.574 INFO:teuthology.orchestra.run.smithi088.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-21T02:01:37.608 INFO:teuthology.orchestra.run.smithi088.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-21T02:01:37.633 INFO:teuthology.orchestra.run.smithi088.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T02:01:37.708 INFO:teuthology.orchestra.run.smithi088.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T02:01:37.917 INFO:teuthology.orchestra.run.smithi088.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T02:01:37.992 INFO:teuthology.orchestra.run.smithi088.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-21T02:01:38.148 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-09-21T02:01:38.149 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-09-21T02:01:38.326 INFO:teuthology.orchestra.run.smithi088.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 5.4 MB/s | 1.8 MB 00:00 2024-09-21T02:01:38.367 INFO:teuthology.orchestra.run.smithi088.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 5.2 MB/s | 2.3 MB 00:00 2024-09-21T02:01:38.393 INFO:teuthology.orchestra.run.smithi088.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 610 kB/s | 41 kB 00:00 2024-09-21T02:01:38.502 INFO:teuthology.orchestra.run.smithi088.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.8 MB/s | 249 kB 00:00 2024-09-21T02:01:38.602 INFO:teuthology.orchestra.run.smithi088.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 1.4 MB 00:00 2024-09-21T02:01:38.919 INFO:teuthology.orchestra.run.smithi088.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.7 MB/s | 2.4 MB 00:00 2024-09-21T02:01:44.416 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-09-21T02:01:44.525 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:01:44.563 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:01:44.616 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:01:44.651 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:01:44.739 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:01:44.972 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:01:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:01:45.452 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:01:45.578 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:01:45.603 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:01:45.703 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:01:45.740 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:01:45.770 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:01:45.928 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:01:45.990 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:01:46.017 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:01:46.123 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:01:46.151 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:01:46.213 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:01:46.235 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:01:46.261 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:01:46.480 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:01:46.630 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:01:46.767 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:01:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:01:46.944 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:01:47.158 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:01:47.267 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:01:47.301 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:01:47.344 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:01:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:01:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:01:47.485 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:01:47.529 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:01:47.561 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:01:47.597 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:01:47.674 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:01:47.736 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:01:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:01:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:01:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:01:47.922 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:01:47.966 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:01:47.994 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:01:48.065 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:01:48.118 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:01:48.192 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:01:48.326 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:01:48.377 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:01:48.915 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:01:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:01:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:01:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:01:49.109 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:01:49.151 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:01:49.307 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:01:49.372 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:01:49.463 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:01:49.527 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:01:49.600 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:01:49.632 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:01:49.668 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:01:49.715 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:01:49.751 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:01:49.794 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:01:49.844 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:01:49.987 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:01:50.029 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:01:50.070 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:01:50.173 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:01:50.422 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:01:50.463 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:01:50.496 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:01:50.520 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:01:50.601 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:01:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:01:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:01:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:01:50.839 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:01:50.878 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:01:50.909 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:01:50.948 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:01:50.983 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:01:51.016 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:01:51.057 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:01:51.092 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:01:51.188 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:01:51.239 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:01:51.270 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:01:51.374 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:01:51.422 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:01:52.141 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:01:52.194 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:01:52.228 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:01:52.328 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:01:52.383 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:01:52.427 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:01:52.514 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:01:52.555 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:01:52.666 INFO:teuthology.orchestra.run.smithi088.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 932 kB/s | 16 MB 00:17 2024-09-21T02:01:52.672 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:01:52.672 INFO:teuthology.orchestra.run.smithi088.stdout:Total 8.2 MB/s | 228 MB 00:27 2024-09-21T02:01:52.737 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:01:52.772 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:01:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:01:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:01:53.611 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:01:53.886 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-09-21T02:01:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-09-21T02:01:54.040 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-09-21T02:01:54.808 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:01:54.926 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:01:54.954 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:01:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:01:55.310 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:01:55.358 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:01:55.448 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:01:55.479 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:01:55.515 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:01:56.012 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:01:56.045 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:01:56.115 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:01:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:01:56.259 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-09-21T02:01:56.260 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-09-21T02:01:58.417 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:01:58.459 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:01:59.027 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:01:59.071 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:01:59.166 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:01:59.166 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:01:59.166 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:01:59.166 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:01:59.206 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:02:02.729 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-09-21T02:02:02.853 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:02:02.892 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:02:02.946 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:02:02.979 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:02:03.013 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:02:03.072 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:02:03.420 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:02:03.539 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:02:03.665 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:02:03.690 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:02:03.781 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:02:03.811 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:02:03.840 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:02:04.017 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:02:04.075 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:02:04.095 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:02:04.183 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:02:04.212 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:02:04.266 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:02:04.296 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:02:04.323 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:02:04.515 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:02:04.684 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:02:04.828 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:02:04.935 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:02:05.071 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:02:05.308 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:02:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:02:05.459 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:02:05.493 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:02:05.556 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:02:05.602 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:02:05.634 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:02:05.677 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:02:05.711 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:02:05.738 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:02:05.815 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:02:05.868 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:02:05.900 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:02:05.944 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:02:06.023 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:02:06.054 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:02:06.097 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:02:06.125 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:02:06.194 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:02:06.248 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:02:06.322 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:02:06.463 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:02:06.517 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:02:07.044 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:02:07.137 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:02:07.167 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:02:07.202 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:02:07.239 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:02:07.281 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:02:07.442 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:02:07.492 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:02:07.583 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:02:07.640 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:02:07.712 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:02:07.744 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:02:07.781 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:02:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:02:07.863 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:02:07.906 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:02:07.948 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:02:08.083 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:02:08.125 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:02:08.166 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:02:08.269 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:02:08.301 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:02:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:02:08.374 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:02:08.398 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:02:08.485 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:02:08.614 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:02:08.647 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:02:08.677 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:02:08.701 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:02:08.740 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:02:08.771 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:02:08.805 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:02:08.831 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:02:08.864 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:02:08.905 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:02:08.940 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:02:09.026 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:02:09.070 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:02:09.102 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:02:09.204 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:02:09.261 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:02:09.972 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:02:10.016 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:02:10.050 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:02:10.147 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:02:10.213 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:02:10.258 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:02:10.345 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:02:10.386 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:02:10.593 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:02:10.616 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:02:10.694 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-09-21T02:02:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:11.293 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:02:11.311 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:02:11.311 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:11.311 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:02:11.311 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:11.323 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:02:11.447 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:02:12.688 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:02:12.765 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:02:12.792 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:02:12.825 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:02:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:02:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:02:12.923 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:12.923 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:02:12.923 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:02:12.923 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:02:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:13.146 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:02:13.155 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:02:13.170 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:02:13.170 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:13.170 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:02:13.170 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:02:13.170 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:02:13.170 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:13.213 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:02:13.304 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:02:13.342 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:02:13.379 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:02:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:02:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:02:13.654 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:13.655 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:02:13.655 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:02:13.655 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:02:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:13.769 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:02:13.825 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:02:13.881 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:02:13.885 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:02:13.916 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:02:13.987 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:02:14.017 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:02:14.064 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:02:14.115 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:02:14.193 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:02:14.280 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:02:14.427 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:02:14.464 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:02:14.505 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:02:14.533 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:02:14.977 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:02:15.007 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:02:15.039 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:02:15.105 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:02:15.138 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:02:16.239 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:02:16.271 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:02:16.838 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:02:16.875 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:02:16.953 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:02:16.954 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:02:16.954 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:02:16.954 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:16.993 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:02:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:02:18.395 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:02:19.356 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:02:19.395 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:02:19.510 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:02:19.678 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:02:19.713 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:02:19.738 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:02:19.738 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:19.738 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:02:19.738 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:02:19.738 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:02:19.738 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:19.789 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:02:19.801 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:02:19.900 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:02:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:02:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:02:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:02:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:02:20.966 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:20.966 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:02:20.966 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:02:20.966 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:02:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:21.009 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:02:21.032 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:02:21.032 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:21.032 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:02:21.032 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:21.084 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:02:21.108 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:02:21.108 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:21.108 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:02:21.108 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:02:21.108 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:02:21.108 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:21.427 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:02:21.450 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:02:21.450 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:21.450 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:02:21.451 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:02:21.451 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:02:21.451 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:25.415 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:02:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:02:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:02:25.554 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:02:25.581 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:02:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:02:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:02:25.858 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:02:25.888 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:02:25.934 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:02:26.520 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:02:26.557 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:02:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:02:27.396 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:02:27.471 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:02:27.511 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:02:27.594 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:02:27.623 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:02:27.649 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:02:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:02:27.683 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:02:27.750 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:02:27.751 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:02:27.843 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:02:27.868 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:02:27.873 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:02:27.873 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:02:28.061 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:02:28.061 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:02:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:02:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /root 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2024-09-21T02:02:28.956 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:29.128 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:02:29.144 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:02:29.144 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:29.144 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:02:29.144 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:30.714 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:02:30.740 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:02:30.740 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:30.741 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:02:30.741 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:02:30.741 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:02:30.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:30.964 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:02:30.987 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:02:30.988 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:30.988 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:02:30.988 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:02:30.988 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:02:30.988 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:31.457 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:02:31.480 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:02:31.480 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:31.480 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:02:31.480 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:02:31.480 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:02:31.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:31.586 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:02:31.635 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:02:31.690 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:02:31.873 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:02:31.923 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:02:31.993 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:02:32.080 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:02:32.236 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:02:32.273 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:02:32.305 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:02:32.334 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:02:32.767 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:02:32.799 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:02:32.831 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:02:32.896 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:02:32.921 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:02:34.738 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:02:34.738 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:02:34.739 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:02:34.740 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:02:34.740 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:02:34.740 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:02:34.752 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:02:34.752 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:02:34.753 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:02:34.754 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:02:34.755 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:02:34.756 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:02:34.757 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:02:34.758 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:02:34.759 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:02:34.760 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:02:34.761 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:02:34.762 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:02:34.763 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:02:34.764 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:02:34.764 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:02:34.764 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:02:36.104 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:02:36.177 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:02:36.991 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:02:36.991 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:02:36.992 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.993 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:02:36.994 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.995 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:02:36.996 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:02:36.997 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:02:36.998 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:02:36.999 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:02:37.000 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:02:37.001 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:37.002 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:02:37.003 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:02:37.004 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:37.004 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-09-21T02:02:37.132 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:02:37.160 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:02:37.271 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:02:37.443 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:02:37.465 DEBUG:teuthology.parallel:result is None 2024-09-21T02:02:37.471 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:02:37.495 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:02:37.495 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:37.495 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:02:37.495 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:02:37.495 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:02:37.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:37.538 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:02:37.549 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:02:37.648 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:02:37.709 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:02:38.673 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:02:38.699 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:02:38.722 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:02:38.722 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:38.722 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:02:38.722 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:02:38.722 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:02:38.722 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:38.766 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:02:38.789 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:02:38.789 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:38.789 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:02:38.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:38.833 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:02:38.856 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:02:38.856 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:38.856 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:02:38.857 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:02:38.857 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:02:38.857 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:02:39.198 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:02:39.198 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:02:39.199 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:02:39.199 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:02:39.199 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:02:39.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:43.230 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:02:43.270 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:02:43.310 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:02:43.343 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:02:43.370 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:02:43.569 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:02:43.617 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:02:43.647 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:02:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:02:43.723 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:02:44.345 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:02:44.388 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:02:44.407 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:02:45.219 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:02:45.300 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:02:45.351 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:02:45.435 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:02:45.470 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:02:45.512 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:02:45.538 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:02:45.538 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:02:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:02:45.622 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:02:45.697 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:02:45.722 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:02:45.727 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:02:45.727 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:02:45.899 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:02:45.899 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:02:45.983 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:02:46.048 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:02:52.512 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:02:52.513 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:02:52.514 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:02:52.514 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:02:52.514 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:02:52.514 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:02:52.514 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:02:52.514 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:02:52.514 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:02:52.526 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:02:52.526 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:02:52.526 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:02:52.526 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:02:52.526 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:02:52.526 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:02:52.527 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:02:52.528 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:02:52.529 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:02:52.530 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:02:52.531 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:02:52.532 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:02:52.533 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:02:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:02:52.535 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:02:52.536 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:02:55.257 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:02:55.257 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:02:55.258 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.259 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:02:55.260 INFO:teuthology.orchestra.run.smithi088.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.261 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:02:55.262 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:02:55.263 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:02:55.264 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:02:55.265 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:02:55.266 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.267 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:02:55.268 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:02:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:02:55.270 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:02:55.270 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:55.270 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-09-21T02:02:55.676 DEBUG:teuthology.parallel:result is None 2024-09-21T02:02:55.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:02:55.828 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:02:55.850 INFO:teuthology.orchestra.run.smithi088.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:02:55.850 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:02:55.851 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:02:55.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:02:56.001 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:02:56.027 INFO:teuthology.orchestra.run.smithi195.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:02:56.027 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:02:56.027 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:02:56.029 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T02:02:56.029 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:56.029 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:02:56.057 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:56.057 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:02:56.099 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T02:02:56.099 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:56.099 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:02:56.124 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:02:56.190 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:56.190 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:02:56.219 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:02:56.286 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T02:02:56.286 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:56.286 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:02:56.311 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:02:56.375 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:56.375 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:02:56.403 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:02:56.470 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T02:02:56.470 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:56.470 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:02:56.494 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:02:56.559 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:56.559 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:02:56.583 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:02:56.648 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T02:02:56.727 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T02:02:56.727 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /var/log/ceph 2024-09-21T02:02:56.729 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-09-21T02:02:56.754 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T02:02:56.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T02:02:56.796 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T02:02:56.821 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T02:02:56.821 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T02:02:56.861 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T02:02:56.896 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T02:02:56.896 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T02:02:56.896 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T02:02:56.896 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:56.896 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T02:02:56.939 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T02:02:57.051 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T02:02:57.089 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:57.089 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T02:02:57.105 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T02:02:57.216 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T02:02:57.246 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T02:02:57.246 INFO:tasks.ceph:config {'conf': {'client': {'client_shutdown_timeout': 120}, 'global': {'ms die on skipped message': False}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T02:02:57.246 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913405', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client_shutdown_timeout': 120}, 'global': {'ms die on skipped message': False}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi088.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFkwIwnUexxNThooSyNon7VqKJErVIzAfege5i1SFcBthAyQJLVHRxzQ1NgNRURc1zxFWNWeJDtrugDkrC95/s=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZcvmmzjbLNJIaQiWTi2nXoZzpBq1KIOhRoAimkx9kJ5qaH+I3I8bkZbnMsS+cYL+mF4Lb8lpwmv7jt3T6e97U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client_shutdown_timeout': 120}, 'global': {'ms die on skipped message': False}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/iozone.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T02:02:57.247 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T02:02:57.249 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T02:02:57.295 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T02:02:57.304 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T02:02:57.360 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:57.360 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:02:57.382 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:02:57.382 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-09-21T02:02:57.437 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:02:57.437 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:57.437 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T02:02:57.438 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:57.438 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:57.438 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 02:02:46.401629473 +0000 2024-09-21T02:02:57.438 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 02:00:26.796723794 +0000 2024-09-21T02:02:57.438 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 02:00:26.796723794 +0000 2024-09-21T02:02:57.438 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 02:00:26.796723794 +0000 2024-09-21T02:02:57.438 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:02:57.500 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T02:02:57.500 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T02:02:57.500 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000206292 s, 2.5 MB/s 2024-09-21T02:02:57.501 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:02:57.557 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-09-21T02:02:57.612 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:02:57.612 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:57.613 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T02:02:57.613 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:57.613 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:57.613 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 02:02:46.401629473 +0000 2024-09-21T02:02:57.613 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 02:00:27.116717383 +0000 2024-09-21T02:02:57.613 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 02:00:27.116717383 +0000 2024-09-21T02:02:57.613 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 02:00:27.116717383 +0000 2024-09-21T02:02:57.613 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:02:57.675 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T02:02:57.676 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T02:02:57.676 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000197814 s, 2.6 MB/s 2024-09-21T02:02:57.677 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:02:57.732 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 02:02:46.402629453 +0000 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 02:00:27.439710911 +0000 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 02:00:27.439710911 +0000 2024-09-21T02:02:57.788 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 02:00:27.439710911 +0000 2024-09-21T02:02:57.789 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:02:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T02:02:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T02:02:57.851 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000176709 s, 2.9 MB/s 2024-09-21T02:02:57.851 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:02:57.908 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 02:02:46.402629453 +0000 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 02:00:27.753704620 +0000 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 02:00:27.753704620 +0000 2024-09-21T02:02:57.963 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 02:00:27.753704620 +0000 2024-09-21T02:02:57.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:02:58.025 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T02:02:58.026 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T02:02:58.026 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.00029203 s, 1.8 MB/s 2024-09-21T02:02:58.027 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:02:58.083 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:02:58.083 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:58.083 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:02:58.100 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:02:58.100 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 02:02:28.570579541 +0000 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 02:00:41.664874641 +0000 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 02:00:41.664874641 +0000 2024-09-21T02:02:58.156 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 02:00:41.664874641 +0000 2024-09-21T02:02:58.157 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:02:58.220 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T02:02:58.220 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T02:02:58.220 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.0889e-05 s, 6.3 MB/s 2024-09-21T02:02:58.221 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:02:58.278 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-09-21T02:02:58.333 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 02:02:28.571579520 +0000 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 02:00:41.967868372 +0000 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 02:00:41.967868372 +0000 2024-09-21T02:02:58.334 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 02:00:41.967868372 +0000 2024-09-21T02:02:58.334 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:02:58.396 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T02:02:58.396 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T02:02:58.397 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000123117 s, 4.2 MB/s 2024-09-21T02:02:58.398 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:02:58.454 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-09-21T02:02:58.509 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 02:02:28.571579520 +0000 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 02:00:42.263862248 +0000 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 02:00:42.263862248 +0000 2024-09-21T02:02:58.510 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 02:00:42.263862248 +0000 2024-09-21T02:02:58.510 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:02:58.571 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T02:02:58.571 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T02:02:58.572 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000106846 s, 4.8 MB/s 2024-09-21T02:02:58.573 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:02:58.629 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-09-21T02:02:58.684 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:02:58.684 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:02:58.684 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T02:02:58.684 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:02:58.684 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:02:58.685 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-21 02:02:28.572579499 +0000 2024-09-21T02:02:58.685 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-21 02:00:42.576855771 +0000 2024-09-21T02:02:58.685 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-21 02:00:42.576855771 +0000 2024-09-21T02:02:58.685 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-09-21 02:00:42.576855771 +0000 2024-09-21T02:02:58.685 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:02:58.747 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-21T02:02:58.747 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-21T02:02:58.747 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.2058e-05 s, 6.2 MB/s 2024-09-21T02:02:58.749 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:02:58.804 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:02:58.804 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T02:02:58.805 INFO:tasks.ceph:Generating config... 2024-09-21T02:02:58.806 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-21T02:02:58.806 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-09-21T02:02:58.806 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T02:02:58.807 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T02:02:58.807 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T02:02:58.807 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T02:02:58.807 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T02:02:58.807 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T02:02:58.807 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T02:02:58.808 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T02:02:58.808 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T02:02:58.809 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T02:02:58.889 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T02:02:58.893 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T02:02:58.946 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:02:58.969 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.88'), ('mon.b', '172.21.15.195'), ('mon.c', '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]')] 2024-09-21T02:02:58.970 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.88,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client_shutdown_timeout': 120}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500}} 2024-09-21T02:02:58.970 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:58.971 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T02:02:59.024 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.88 --add b 172.21.15.195 --addv c '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: generated fsid 0ee3c2c3-8406-48b8-a5fb-ec622d976411 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:setting min_mon_release = quincy 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 0 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:fsid 0ee3c2c3-8406-48b8-a5fb-ec622d976411 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:last_changed 2024-09-21T02:02:59.122435+0000 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-09-21T02:02:59.122435+0000 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:min_mon_release 17 (quincy) 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:election_strategy: 1 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-09-21T02:02:59.122 INFO:teuthology.orchestra.run.smithi088.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-09-21T02:02:59.123 INFO:teuthology.orchestra.run.smithi088.stdout:2: [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] mon.c 2024-09-21T02:02:59.123 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T02:02:59.124 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T02:02:59.138 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0ee3c2c3-8406-48b8-a5fb-ec622d976411... 2024-09-21T02:02:59.139 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T02:02:59.181 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T02:02:59.222 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: chdir = "" 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: auth supported = cephx 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: filestore xattr use omap = true 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: mon clock drift allowed = 1.000 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: osd crush chooseleaf type = 0 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: auth debug = true 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on old message = true 2024-09-21T02:02:59.223 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on bug = true 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: mon pg warn max object skew = 0 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default size = 2 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow primary affinity = true 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow pg remap = true 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on legacy crush tunables = false 2024-09-21T02:02:59.224 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on crush straw calc version zero = false 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on no sortbitwise = false 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on osd down out interval zero = false 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on too few osds = false 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: mon_allow_pool_size_one = true 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: osd default data pool replay window = 5 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.225 INFO:teuthology.orchestra.run.smithi088.stdout: mon allow pool delete = true 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: mon cluster log file level = debug 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: debug asserts on shutdown = true 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: mon health detail to clog = false 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: mon host = "172.21.15.88,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on skipped message = False 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = 0ee3c2c3-8406-48b8-a5fb-ec622d976411 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout:[osd] 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: osd journal size = 100 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub load threshold = 5.0 2024-09-21T02:02:59.226 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub max interval = 600 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd mclock profile = high_recovery_ops 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd recover clone overlap = true 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd recovery max chunk = 1048576 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug shutdown = true 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug op order = true 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug verify stray on activate = true 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug trim objects = true 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd open classes on start = true 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug pg log writeout = true 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.227 INFO:teuthology.orchestra.run.smithi088.stdout: osd deep scrub update digest min age = 30 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: osd map max advance = 10 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: journal zero on create = true 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: filestore ondisk finisher threads = 3 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: filestore apply finisher threads = 3 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: bdev debug aio = true 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug misdirected ops = true 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: bdev async discard = True 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: bdev enable discard = True 2024-09-21T02:02:59.228 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore allocator = bitmap 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore block size = 96636764160 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore fsck on mount = True 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluefs = 1/20 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluestore = 1/20 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: debug osd = 20 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: debug rocksdb = 4/10 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: mon client directed command retry = 5 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd full ratio = 0.9 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd nearfull ratio = 0.8 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject delay max = 1 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject delay probability = 0.005 2024-09-21T02:02:59.229 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject delay type = client mds 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject socket failures = 2500 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: osd blocked scrub grace period = 3600 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: osd failsafe full ratio = 0.95 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: osd max markdown count = 1000 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: osd objectstore = bluestore 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr] 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: debug mgr = 20 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:02:59.230 INFO:teuthology.orchestra.run.smithi088.stdout: mgr/telemetry/nag = false 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout:[mon] 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: debug paxos = 20 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: mon data avail warn = 5 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: mon mgr mkfs grace = 240 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd reporter subtree level = osd 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd prime pg temp = true 2024-09-21T02:02:59.231 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: # 1m isn't quite enough 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: mon_down_mkfs_grace = 2m 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_filestore_osds = false 2024-09-21T02:02:59.232 INFO:teuthology.orchestra.run.smithi088.stdout: mon down mkfs grace = 300 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout:[client] 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout: rgw cache enabled = true 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable ops log = true 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable usage log = true 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout: client_shutdown_timeout = 120 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.a] 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.b] 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.c] 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-09-21T02:02:59.233 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.e] 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.f] 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout:[mds] 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout: mds oft prefetch dirfrags = True 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout: mds_sleep_rank_change = 5000000.0 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout: mon client directed command retry = 5 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject delay max = 1 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject delay probability = 0.005 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject delay type = client mds osd 2024-09-21T02:02:59.234 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject socket failures = 2500 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.235 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.236 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.237 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.88,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on skipped message = False 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 0ee3c2c3-8406-48b8-a5fb-ec622d976411 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.238 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.239 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-09-21T02:02:59.240 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: mon client directed command retry = 5 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject delay max = 1 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject delay probability = 0.005 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject delay type = client mds 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject socket failures = 2500 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: osd blocked scrub grace period = 3600 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: osd max markdown count = 1000 2024-09-21T02:02:59.241 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-21T02:02:59.242 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.243 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: mon down mkfs grace = 300 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T02:02:59.244 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout: client_shutdown_timeout = 120 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.e] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.f] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout: mds oft prefetch dirfrags = True 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout: mds_sleep_rank_change = 5000000.0 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout: mon client directed command retry = 5 2024-09-21T02:02:59.245 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject delay max = 1 2024-09-21T02:02:59.246 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject delay probability = 0.005 2024-09-21T02:02:59.246 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject delay type = client mds osd 2024-09-21T02:02:59.246 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject socket failures = 2500 2024-09-21T02:02:59.247 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T02:02:59.247 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T02:02:59.352 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T02:02:59.352 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:59.352 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T02:02:59.367 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:59.368 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T02:02:59.422 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:02:59.422 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:59.422 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T02:02:59.422 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:02:59.497 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:02:59.497 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:02:59.553 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:02:59.553 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:59.554 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T02:02:59.554 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:02:59.589 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:02:59.589 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:02:59.645 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T02:02:59.645 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T02:02:59.645 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T02:02:59.706 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T02:02:59.709 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T02:02:59.779 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T02:02:59.782 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T02:02:59.782 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T02:02:59.842 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T02:02:59.846 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T02:02:59.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T02:02:59.971 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T02:02:59.973 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T02:02:59.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-21T02:03:00.098 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-21T02:03:00.101 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-21T02:03:00.125 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T02:03:00.186 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T02:03:00.189 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T02:03:00.213 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T02:03:00.314 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T02:03:00.317 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T02:03:00.341 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-21T02:03:00.441 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-21T02:03:00.445 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-21T02:03:00.469 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T02:03:00.469 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T02:03:00.520 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T02:03:00.533 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T02:03:00.533 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T02:03:00.534 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T02:03:00.558 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:00.558 INFO:tasks.ceph:role: osd.0 2024-09-21T02:03:00.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:03:00.558 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T02:03:00.629 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:00.630 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:01.134 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T02:03:01.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T02:03:01.136 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T02:03:01.272 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T02:03:01.304 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T02:03:01.370 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:01.371 INFO:tasks.ceph:role: osd.1 2024-09-21T02:03:01.371 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:03:01.371 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T02:03:01.443 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:01.443 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:01.443 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:01.443 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:01.443 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:01.443 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:01.444 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:01.444 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:01.444 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:01.444 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:01.893 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T02:03:01.895 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T02:03:01.895 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T02:03:01.979 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T02:03:02.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T02:03:02.074 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:02.075 INFO:tasks.ceph:role: osd.2 2024-09-21T02:03:02.075 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:03:02.075 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T02:03:02.147 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:02.148 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:02.631 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T02:03:02.633 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T02:03:02.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T02:03:02.755 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T02:03:02.786 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T02:03:02.853 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:02.853 INFO:tasks.ceph:role: osd.3 2024-09-21T02:03:02.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:03:02.853 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T02:03:02.927 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:02.927 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:02.927 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:02.927 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:02.927 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:02.928 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:02.928 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:02.928 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:02.928 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:02.928 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:03.422 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T02:03:03.425 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T02:03:03.425 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T02:03:03.554 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T02:03:03.588 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:03.703+0000 7faababfd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T02:03:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:03.703+0000 7faababfd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T02:03:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:03.703+0000 7faababfd640 -1 bdev(0x555b124a5900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:03.703+0000 7faababfd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T02:03:06.617 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T02:03:06.646 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:06.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:06.746+0000 7f06e3eb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T02:03:06.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:06.747+0000 7f06e3eb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T02:03:06.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:06.747+0000 7f06e3eb2640 -1 bdev(0x55ad36fdd900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:06.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:06.747+0000 7f06e3eb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T02:03:09.664 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T02:03:09.693 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:09.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:09.795+0000 7fa1019dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T02:03:09.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:09.795+0000 7fa1019dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T02:03:09.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:09.795+0000 7fa1019dd640 -1 bdev(0x55996c94d900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:09.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:09.795+0000 7fa1019dd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T02:03:12.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T02:03:12.992 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:13.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:13.094+0000 7f08186b4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T02:03:13.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:13.094+0000 7f08186b4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T02:03:13.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:13.094+0000 7f08186b4640 -1 bdev(0x55d2a2455900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:13.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:13.095+0000 7f08186b4640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T02:03:16.017 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T02:03:16.046 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T02:03:16.077 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:16.077 INFO:tasks.ceph:role: osd.4 2024-09-21T02:03:16.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:03:16.077 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:16.145 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:16.177 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T02:03:16.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T02:03:16.180 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T02:03:16.251 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T02:03:16.322 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T02:03:16.387 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:16.387 INFO:tasks.ceph:role: osd.5 2024-09-21T02:03:16.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:03:16.388 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T02:03:16.451 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:16.452 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:16.452 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:16.453 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:16.453 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:16.453 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:16.454 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:16.454 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:16.454 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:16.454 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:16.484 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T02:03:16.486 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T02:03:16.486 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T02:03:16.555 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T02:03:16.624 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T02:03:16.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:16.689 INFO:tasks.ceph:role: osd.6 2024-09-21T02:03:16.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:03:16.689 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T02:03:16.753 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:16.754 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:16.787 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T02:03:16.789 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T02:03:16.789 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T02:03:16.858 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T02:03:16.928 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T02:03:16.992 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:03:16.992 INFO:tasks.ceph:role: osd.7 2024-09-21T02:03:16.992 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:03:16.992 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T02:03:17.056 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:03:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:03:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:03:17.056 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T02:03:17.057 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:03:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:03:17.057 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:03:17.057 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T02:03:17.057 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:03:17.057 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:03:17.089 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-21T02:03:17.090 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-21T02:03:17.091 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T02:03:17.159 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T02:03:17.227 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:17.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:17.336+0000 7fd29af89640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T02:03:17.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:17.336+0000 7fd29af89640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T02:03:17.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:17.336+0000 7fd29af89640 -1 bdev(0x55e29a823900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:17.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:17.336+0000 7fd29af89640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T02:03:20.253 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T02:03:20.282 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:20.383 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:20.383+0000 7f1d4841c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T02:03:20.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:20.383+0000 7f1d4841c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T02:03:20.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:20.383+0000 7f1d4841c640 -1 bdev(0x55e905101900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:20.384 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:20.383+0000 7f1d4841c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T02:03:23.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T02:03:23.325 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:23.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:23.428+0000 7f452a60d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T02:03:23.429 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:23.428+0000 7f452a60d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T02:03:23.429 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:23.428+0000 7f452a60d640 -1 bdev(0x55b5b4205900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:23.429 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:23.428+0000 7f452a60d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T02:03:26.340 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T02:03:26.371 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:26.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:26.473+0000 7fb094a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T02:03:26.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:26.473+0000 7fb094a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T02:03:26.474 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:26.473+0000 7fb094a0f640 -1 bdev(0x561066d41900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T02:03:26.475 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:26.473+0000 7fb094a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T02:03:27.248 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:03:27.251 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:03:29.386 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T02:03:29.412 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T02:03:29.412 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.412 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T02:03:29.442 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.443 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T02:03:29.511 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.511 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T02:03:29.577 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.577 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-21T02:03:29.644 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.645 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T02:03:29.711 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.711 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T02:03:29.778 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.778 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T02:03:29.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:29.843 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T02:03:29.908 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:29.908 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T02:03:29.937 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:29.937 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T02:03:30.005 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:30.005 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T02:03:30.071 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:30.071 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-21T02:03:30.137 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:30.137 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T02:03:30.203 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:30.203 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T02:03:30.269 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:30.269 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T02:03:30.335 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:30.335 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T02:03:30.399 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:30.399 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T02:03:30.417 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T02:03:30.417 DEBUG:teuthology.orchestra.run.smithi088:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T02:03:30.459 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T02:03:30.482 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.y] 2024-09-21T02:03:30.482 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBTKe5m0icgKhAA3edEtBXUOgCkSbj1g9xg5w== 2024-09-21T02:03:30.482 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-09-21T02:03:30.482 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBTKe5mJp9EMhAApM/ezEzuzteQROUaBYoqxQ== 2024-09-21T02:03:30.482 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-09-21T02:03:30.482 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBTKe5mBbLhORAAebHATM394NVref6Cc99wOw== 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.e] 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBUKe5mjtTlBRAA1isv3+jAsFsVjyog6jsqQw== 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.0] 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBXKe5mjBv7KRAALNwo82vKb4Ao8PrQtf7hXA== 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.1] 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBaKe5m+2iWLBAAQcBZ+fVl6t2aVNgrp+Mxvg== 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.2] 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBdKe5mOOJsLxAALYM9jJ0Iyh6zVgZqWgSdGQ== 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.3] 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBhKe5mS+KnBRAAkzkAyYE3L/6kFfV6WngMCQ== 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.x] 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBTKe5mqa52LhAAFrjPvwc1/m2HtVsw4o9MOQ== 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-09-21T02:03:30.483 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBUKe5mexoiCxAAmZrjb0069dWiV2ergIj1Mg== 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBUKe5mGV++EhAAZ4gimisuYi+6aaxZu8m2Xw== 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.f] 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBUKe5m5bldGhAAL3Qq4OCCGxib47n1cB0fbA== 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.4] 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBlKe5mp/gVFBAASV0vR5jq8nMqapGhSnQlfA== 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.5] 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBoKe5mPTnjFhAAcCdMe+frzrwCcxyApTcBzg== 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.6] 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBrKe5mFkORGRAAbmbcozcngceu1mA0DTX0lg== 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.7] 2024-09-21T02:03:30.484 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBuKe5mmXdAHBAA14GkbSpqQtGqWvR/9CfpeA== 2024-09-21T02:03:30.485 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T02:03:30.485 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBUKe5msqwNHxAAxHN/hPMpLlENwadSD2dgVQ== 2024-09-21T02:03:30.486 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2024-09-21T02:03:30.486 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBTKe5m0icgKhAA3edEtBXUOgCkSbj1g9xg5w== 2024-09-21T02:03:30.486 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-09-21T02:03:30.486 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBTKe5mJp9EMhAApM/ezEzuzteQROUaBYoqxQ== 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBTKe5mBbLhORAAebHATM394NVref6Cc99wOw== 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.e] 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBUKe5mjtTlBRAA1isv3+jAsFsVjyog6jsqQw== 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBXKe5mjBv7KRAALNwo82vKb4Ao8PrQtf7hXA== 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBaKe5m+2iWLBAAQcBZ+fVl6t2aVNgrp+Mxvg== 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBdKe5mOOJsLxAALYM9jJ0Iyh6zVgZqWgSdGQ== 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBhKe5mS+KnBRAAkzkAyYE3L/6kFfV6WngMCQ== 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2024-09-21T02:03:30.487 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBTKe5mqa52LhAAFrjPvwc1/m2HtVsw4o9MOQ== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBUKe5mexoiCxAAmZrjb0069dWiV2ergIj1Mg== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBUKe5mGV++EhAAZ4gimisuYi+6aaxZu8m2Xw== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.f] 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBUKe5m5bldGhAAL3Qq4OCCGxib47n1cB0fbA== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBlKe5mp/gVFBAASV0vR5jq8nMqapGhSnQlfA== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBoKe5mPTnjFhAAcCdMe+frzrwCcxyApTcBzg== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBrKe5mFkORGRAAbmbcozcngceu1mA0DTX0lg== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBuKe5mmXdAHBAA14GkbSpqQtGqWvR/9CfpeA== 2024-09-21T02:03:30.488 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2024-09-21T02:03:30.489 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBUKe5msqwNHxAAxHN/hPMpLlENwadSD2dgVQ== 2024-09-21T02:03:30.489 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:03:30.529 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:03:30.592 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:30.595 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:30.659 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:30.661 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:30.725 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:30.727 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:30.791 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:30.793 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:30.854 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:30.856 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:30.918 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:30.920 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:30.982 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:30.984 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.045 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:03:31.048 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:03:31.109 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:31.111 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:31.175 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:31.177 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:31.241 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:31.244 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:03:31.308 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.310 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.371 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.373 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.434 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.437 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.499 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.503 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:03:31.565 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:03:31.567 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:03:31.631 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T02:03:31.632 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T02:03:31.656 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:03:31.763 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T02:03:31.787 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T02:03:31.812 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:03:31.919 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T02:03:31.943 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T02:03:32.008 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:03:32.114 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T02:03:32.139 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:32.141 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:03:32.194 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T02:03:32.194 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T02:03:32.195 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T02:03:32.197 INFO:tasks.ceph.mon.a:Started 2024-09-21T02:03:32.197 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T02:03:32.197 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T02:03:32.237 INFO:tasks.ceph.mon.b:Started 2024-09-21T02:03:32.237 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T02:03:32.237 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T02:03:32.239 INFO:tasks.ceph.mon.c:Started 2024-09-21T02:03:32.239 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T02:03:32.239 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T02:03:32.239 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T02:03:32.241 INFO:tasks.ceph.mgr.y:Started 2024-09-21T02:03:32.241 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T02:03:32.241 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T02:03:32.243 INFO:tasks.ceph.mgr.x:Started 2024-09-21T02:03:32.243 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T02:03:32.244 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T02:03:32.245 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T02:03:32.246 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T02:03:32.288 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T02:03:32.288 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T02:03:32.332 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T02:03:32.333 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T02:03:32.376 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T02:03:32.376 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T02:03:32.419 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T02:03:32.419 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T02:03:32.461 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T02:03:32.462 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T02:03:32.505 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T02:03:32.506 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T02:03:32.549 INFO:tasks.ceph:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T02:03:32.549 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T02:03:32.595 INFO:tasks.ceph:setting config [mds] mds_bal_merge_size = 5 2024-09-21T02:03:32.595 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_bal_merge_size 5 2024-09-21T02:03:32.641 INFO:tasks.ceph:setting config [mds] mds_bal_split_bits = 3 2024-09-21T02:03:32.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_bal_split_bits 3 2024-09-21T02:03:32.685 INFO:tasks.ceph:setting config [mds] mds_bal_split_size = 100 2024-09-21T02:03:32.685 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_bal_split_size 100 2024-09-21T02:03:32.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.691+0000 7fc562ffd640 1 -- 172.21.15.88:0/3237810535 <== mon.1 v2:172.21.15.195:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v3)=0 v3) ==== 109+0+0 (secure 0 0 0) 0x7fc564019900 con 0x7fc5740cdee0 2024-09-21T02:03:32.695 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T02:03:32.695 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T02:03:32.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.698+0000 7fbb9898a640 1 -- 172.21.15.88:0/1670306903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c97d0 msgr2=0x7fbb94156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.698+0000 7fbb9898a640 1 --2- 172.21.15.88:0/1670306903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c97d0 0x7fbb94156d60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb940cbb50 tx=0x7fbb8402aa20 comp rx=0 tx=0).stop 2024-09-21T02:03:32.700 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T02:03:32.700 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T02:03:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.698+0000 7fbb9898a640 1 -- 172.21.15.88:0/1670306903 shutdown_connections 2024-09-21T02:03:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.699+0000 7fbb9898a640 1 --2- 172.21.15.88:0/1670306903 >> v2:172.21.15.195:3300/0 conn(0x7fbb94159d70 0x7fbb9415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.699+0000 7fbb9898a640 1 --2- 172.21.15.88:0/1670306903 >> v2:172.21.15.88:3300/0 conn(0x7fbb940c72b0 0x7fbb940c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.699+0000 7fbb9898a640 1 --2- 172.21.15.88:0/1670306903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c97d0 0x7fbb94156d60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.699+0000 7fbb9898a640 1 -- 172.21.15.88:0/1670306903 >> 172.21.15.88:0/1670306903 conn(0x7fbb940720b0 msgr2=0x7fbb940cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.699+0000 7fbb9898a640 1 -- 172.21.15.88:0/1670306903 shutdown_connections 2024-09-21T02:03:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.699+0000 7fbb9898a640 1 -- 172.21.15.88:0/1670306903 wait complete. 2024-09-21T02:03:32.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.702+0000 7fc578d6e640 1 -- 172.21.15.88:0/3237810535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5740cdee0 msgr2=0x7fc5741ecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.702+0000 7fc578d6e640 1 --2- 172.21.15.88:0/3237810535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5740cdee0 0x7fc5741ecf60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc564009910 tx=0x7fc564009dd0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.704 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T02:03:32.704 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T02:03:32.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fc578d6e640 1 -- 172.21.15.88:0/3237810535 shutdown_connections 2024-09-21T02:03:32.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fc578d6e640 1 --2- 172.21.15.88:0/3237810535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5740c8af0 0x7fc57413fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fc578d6e640 1 --2- 172.21.15.88:0/3237810535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc574151e10 0x7fc574148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fc578d6e640 1 --2- 172.21.15.88:0/3237810535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5740cdee0 0x7fc5741ecf60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fc578d6e640 1 -- 172.21.15.88:0/3237810535 >> 172.21.15.88:0/3237810535 conn(0x7fc574071570 msgr2=0x7fc5740c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fbb9898a640 1 Processor -- start 2024-09-21T02:03:32.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fbb9898a640 1 -- start start 2024-09-21T02:03:32.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fc578d6e640 1 -- 172.21.15.88:0/3237810535 shutdown_connections 2024-09-21T02:03:32.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.703+0000 7fc578d6e640 1 -- 172.21.15.88:0/3237810535 wait complete. 2024-09-21T02:03:32.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.706+0000 7f54f4ff9640 1 -- 172.21.15.88:0/2061340518 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54ec042070 con 0x7f54f80c28d0 2024-09-21T02:03:32.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.706+0000 7f54f4ff9640 1 -- 172.21.15.88:0/2061340518 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54ec03b460 con 0x7f54f80c28d0 2024-09-21T02:03:32.711 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T02:03:32.711 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T02:03:32.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.709+0000 7f54e67fc640 1 -- 172.21.15.88:0/2061340518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d4005080 con 0x7f54f80c28d0 2024-09-21T02:03:32.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.713+0000 7f6ae6ffd640 1 -- 172.21.15.88:0/1021690189 <== mon.0 v2:172.21.15.88:3300/0 9 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7f6ae800fc40 con 0x7f6af8152c00 2024-09-21T02:03:32.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.712+0000 7fd8627fc640 1 -- 172.21.15.88:0/757198136 <== mon.0 v2:172.21.15.88:3300/0 9 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7fd868003be0 con 0x7fd86c0cd9e0 2024-09-21T02:03:32.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.714+0000 7f54f4ff9640 1 -- 172.21.15.88:0/2061340518 <== mon.0 v2:172.21.15.88:3300/0 6 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7f54ec003f20 con 0x7f54f80c28d0 2024-09-21T02:03:32.719 INFO:tasks.ceph:setting config [mds] mds_dir_prefetch = True 2024-09-21T02:03:32.719 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_dir_prefetch True 2024-09-21T02:03:32.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.717+0000 7fb110ff9640 1 -- 172.21.15.88:0/1411202026 <== mon.0 v2:172.21.15.88:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v4)=0 v4) ==== 123+0+0 (secure 0 0 0) 0x7fb10c025440 con 0x7fb1141557c0 2024-09-21T02:03:32.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.717+0000 7fb110ff9640 1 -- 172.21.15.88:0/1411202026 <== mon.0 v2:172.21.15.88:3300/0 10 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7fb10c0256e0 con 0x7fb1141557c0 2024-09-21T02:03:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.721+0000 7fbb9898a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c72b0 0x7fbb94155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.721+0000 7fbb9898a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb940c97d0 0x7fbb9414ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.721+0000 7fbb9898a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb94159d70 0x7fbb941e2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb9898a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94159850 con 0x7fbb94159d70 2024-09-21T02:03:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb9898a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbb940c3270 con 0x7fbb940c97d0 2024-09-21T02:03:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb9898a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbb940c9d00 con 0x7fbb940c72b0 2024-09-21T02:03:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 -- 172.21.15.88:0/1411202026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1141557c0 msgr2=0x7fb1141540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 --2- 172.21.15.88:0/1411202026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1141557c0 0x7fb1141540e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb10c001b10 tx=0x7fb10c006720 comp rx=0 tx=0).stop 2024-09-21T02:03:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 -- 172.21.15.88:0/1411202026 shutdown_connections 2024-09-21T02:03:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 --2- 172.21.15.88:0/1411202026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1141557c0 0x7fb1141540e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 --2- 172.21.15.88:0/1411202026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1141ee8d0 0x7fb114153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 --2- 172.21.15.88:0/1411202026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0fc0037a0 0x7fb1141463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 -- 172.21.15.88:0/1411202026 >> 172.21.15.88:0/1411202026 conn(0x7fb114071570 msgr2=0x7fb1140cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.730 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T02:03:32.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb91d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb940c97d0 0x7fbb9414ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb91d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb940c97d0 0x7fbb9414ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:47080/0 (socket says 172.21.15.88:47080) 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb91d9b640 1 -- 172.21.15.88:0/2875667756 learned_addr learned my addr 172.21.15.88:0/2875667756 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb9259c640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c72b0 0x7fbb94155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb91d9b640 1 -- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c72b0 msgr2=0x7fbb94155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb91d9b640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c72b0 0x7fbb94155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb91d9b640 1 -- 172.21.15.88:0/2875667756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb94159d70 msgr2=0x7fbb941e2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fbb91d9b640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb94159d70 0x7fbb941e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.723+0000 7fbb91d9b640 1 -- 172.21.15.88:0/2875667756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941e5990 con 0x7fbb940c97d0 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.724+0000 7fbb91d9b640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb940c97d0 0x7fbb9414ca50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb880058c0 tx=0x7fbb8800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.728+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88017020 con 0x7fbb940c97d0 2024-09-21T02:03:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.728+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(3 keys) ==== 85+0+0 (secure 0 0 0) 0x7fbb88005230 con 0x7fbb940c97d0 2024-09-21T02:03:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.728+0000 7fbb9898a640 1 -- 172.21.15.88:0/2875667756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb94157ee0 con 0x7fbb940c97d0 2024-09-21T02:03:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.728+0000 7fbb9898a640 1 -- 172.21.15.88:0/2875667756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb94154990 con 0x7fbb940c97d0 2024-09-21T02:03:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 -- 172.21.15.88:0/1411202026 shutdown_connections 2024-09-21T02:03:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.722+0000 7fb11a3f1640 1 -- 172.21.15.88:0/1411202026 wait complete. 2024-09-21T02:03:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.730+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88005600 con 0x7fbb940c97d0 2024-09-21T02:03:32.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.732+0000 7fbb80ff9640 1 -- 172.21.15.88:0/2875667756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb78005080 con 0x7fbb940c97d0 2024-09-21T02:03:32.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.733+0000 7f54f4ff9640 1 -- 172.21.15.88:0/2061340518 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f54ec005410 con 0x7f54f80c28d0 2024-09-21T02:03:32.736 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T02:03:32.736 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T02:03:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.733+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbb8801a020 con 0x7fbb940c97d0 2024-09-21T02:03:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.733+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbb88010540 con 0x7fbb940c97d0 2024-09-21T02:03:32.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.742+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbb88021020 con 0x7fbb940c97d0 2024-09-21T02:03:32.748 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T02:03:32.748 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T02:03:32.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.751+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fbb88009c20 con 0x7fbb940c97d0 2024-09-21T02:03:32.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.761+0000 7f6ae6ffd640 1 -- 172.21.15.88:0/1021690189 <== mon.0 v2:172.21.15.88:3300/0 10 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v5)=0 v5) ==== 114+0+0 (secure 0 0 0) 0x7f6ae801b460 con 0x7f6af8152c00 2024-09-21T02:03:32.766 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T02:03:32.766 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T02:03:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.763+0000 7f6ae4ff9640 1 -- 172.21.15.88:0/1021690189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af8152c00 msgr2=0x7f6af8157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.763+0000 7f6ae4ff9640 1 --2- 172.21.15.88:0/1021690189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af8152c00 0x7f6af8157e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006fd0 tx=0x7f6ae80039c0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.764+0000 7fd8627fc640 1 -- 172.21.15.88:0/757198136 <== mon.0 v2:172.21.15.88:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd868031400 con 0x7fd86c0cd9e0 2024-09-21T02:03:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.766+0000 7f6ae4ff9640 1 -- 172.21.15.88:0/1021690189 shutdown_connections 2024-09-21T02:03:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.766+0000 7f6ae4ff9640 1 --2- 172.21.15.88:0/1021690189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6af81e3ac0 0x7f6af81e3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.766+0000 7f6ae4ff9640 1 --2- 172.21.15.88:0/1021690189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6af8156a00 0x7f6af81f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.766+0000 7f6ae4ff9640 1 --2- 172.21.15.88:0/1021690189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af8152c00 0x7f6af8157e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.766+0000 7f6ae4ff9640 1 -- 172.21.15.88:0/1021690189 >> 172.21.15.88:0/1021690189 conn(0x7f6af8071570 msgr2=0x7f6af80c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.767+0000 7f6ae4ff9640 1 -- 172.21.15.88:0/1021690189 shutdown_connections 2024-09-21T02:03:32.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.767+0000 7f6ae4ff9640 1 -- 172.21.15.88:0/1021690189 wait complete. 2024-09-21T02:03:32.771 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T02:03:32.771 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T02:03:32.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.764+0000 7f54f4ff9640 1 -- 172.21.15.88:0/2061340518 <== mon.0 v2:172.21.15.88:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f54ec004d50 con 0x7f54f80c28d0 2024-09-21T02:03:32.774 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T02:03:32.774 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T02:03:32.787 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T02:03:32.787 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T02:03:32.830 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T02:03:32.830 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T02:03:32.839 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T02:03:32.840 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T02:03:32.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.861+0000 7f320359e640 1 -- 172.21.15.88:0/2333028042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc005c00 msgr2=0x7f31fc006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.861+0000 7f320359e640 1 --2- 172.21.15.88:0/2333028042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc005c00 0x7f31fc006010 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009850 tx=0x7f31f402a540 comp rx=0 tx=0).stop 2024-09-21T02:03:32.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.862+0000 7f320359e640 1 -- 172.21.15.88:0/2333028042 shutdown_connections 2024-09-21T02:03:32.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.862+0000 7f320359e640 1 --2- 172.21.15.88:0/2333028042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc005c00 0x7f31fc006010 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.862+0000 7f320359e640 1 --2- 172.21.15.88:0/2333028042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc0df620 0x7f31fc0454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.862+0000 7f320359e640 1 -- 172.21.15.88:0/2333028042 >> 172.21.15.88:0/2333028042 conn(0x7f31fc04fe30 msgr2=0x7f31fc050240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.864 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T02:03:32.864 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T02:03:32.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.863+0000 7f320359e640 1 -- 172.21.15.88:0/2333028042 shutdown_connections 2024-09-21T02:03:32.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.863+0000 7f320359e640 1 -- 172.21.15.88:0/2333028042 wait complete. 2024-09-21T02:03:32.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.866+0000 7f320359e640 1 Processor -- start 2024-09-21T02:03:32.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.866+0000 7f320359e640 1 -- start start 2024-09-21T02:03:32.870 DEBUG:tasks.ceph:set 27 configs 2024-09-21T02:03:32.870 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi088') 2024-09-21T02:03:32.870 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi088') 2024-09-21T02:03:32.870 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi088') 2024-09-21T02:03:32.870 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T02:03:32.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.868+0000 7f320359e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc044d20 0x7f31fc0d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.868+0000 7f320359e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0df620 0x7f31fc049860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.868+0000 7f320359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31fc0d0a70 0x7f31fc04ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.869+0000 7f3201d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0df620 0x7f31fc049860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.869+0000 7f3201d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0df620 0x7f31fc049860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:47116/0 (socket says 172.21.15.88:47116) 2024-09-21T02:03:32.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.896+0000 7f3202d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31fc0d0a70 0x7f31fc04ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.896+0000 7f3202d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31fc0d0a70 0x7f31fc04ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41090/0 (socket says 172.21.15.88:41090) 2024-09-21T02:03:32.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.896+0000 7f320359e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0465d0 con 0x7f31fc0d0a70 2024-09-21T02:03:32.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.896+0000 7f320359e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc006550 con 0x7f31fc0df620 2024-09-21T02:03:32.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.896+0000 7f320359e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc0468b0 con 0x7f31fc044d20 2024-09-21T02:03:32.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.899+0000 7f3201d9b640 1 -- 172.21.15.88:0/2311165417 learned_addr learned my addr 172.21.15.88:0/2311165417 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:32.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.899+0000 7f320259c640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc044d20 0x7f31fc0d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.899+0000 7f3201d9b640 1 -- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc044d20 msgr2=0x7f31fc0d4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.899+0000 7f3201d9b640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc044d20 0x7f31fc0d4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.899+0000 7f3201d9b640 1 -- 172.21.15.88:0/2311165417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31fc0d0a70 msgr2=0x7f31fc04ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.899+0000 7f3201d9b640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31fc0d0a70 0x7f31fc04ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.899+0000 7f3201d9b640 1 -- 172.21.15.88:0/2311165417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc0d6f70 con 0x7f31fc0df620 2024-09-21T02:03:32.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.900+0000 7f3201d9b640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0df620 0x7f31fc049860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f31f800e7c0 tx=0x7f31f800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:32.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.902+0000 7f31ca132640 1 -- 172.21.15.88:0/2558952591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31ac003ad0 msgr2=0x7f31ac003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.902+0000 7f31ca132640 1 --2- 172.21.15.88:0/2558952591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31ac003ad0 0x7f31ac003f00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f31c40c3200 tx=0x7f31ac005830 comp rx=0 tx=0).stop 2024-09-21T02:03:32.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.903+0000 7f31c3fff640 1 -- 172.21.15.88:0/2558952591 reap_dead start 2024-09-21T02:03:32.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.903+0000 7f31ca132640 1 -- 172.21.15.88:0/2558952591 shutdown_connections 2024-09-21T02:03:32.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.903+0000 7f31ca132640 1 -- 172.21.15.88:0/2558952591 >> 172.21.15.88:0/2558952591 conn(0x7f31c406d390 msgr2=0x7f31c40ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.903+0000 7f31ca132640 1 -- 172.21.15.88:0/2558952591 shutdown_connections 2024-09-21T02:03:32.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.905+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f800ccc0 con 0x7f31fc0df620 2024-09-21T02:03:32.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.903+0000 7f31ca132640 1 -- 172.21.15.88:0/2558952591 wait complete. 2024-09-21T02:03:32.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.906+0000 7f31ca132640 1 Processor -- start 2024-09-21T02:03:32.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.905+0000 7f320359e640 1 -- 172.21.15.88:0/2311165417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc186020 con 0x7f31fc0df620 2024-09-21T02:03:32.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.905+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f31f8005020 con 0x7f31fc0df620 2024-09-21T02:03:32.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.905+0000 7f320359e640 1 -- 172.21.15.88:0/2311165417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc185d40 con 0x7f31fc0df620 2024-09-21T02:03:32.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.905+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8005430 con 0x7f31fc0df620 2024-09-21T02:03:32.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.906+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31f8004b30 con 0x7f31fc0df620 2024-09-21T02:03:32.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.906+0000 7f31ca132640 1 -- start start 2024-09-21T02:03:32.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.906+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f31f8018430 con 0x7f31fc0df620 2024-09-21T02:03:32.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31ca132640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c41f62f0 0x7f31c4150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31ca132640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c4155590 0x7f31c415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31ca132640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c4150eb0 0x7f31c415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31ca132640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31c4072bf0 con 0x7f31c41f62f0 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31ca132640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f31c4075a10 con 0x7f31c4150eb0 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31ca132640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f31c415bc60 con 0x7f31c4155590 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31c37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c41f62f0 0x7f31c4150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31c37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c41f62f0 0x7f31c4150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41102/0 (socket says 172.21.15.88:41102) 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.909+0000 7f31c37fe640 1 -- 172.21.15.88:0/914603780 learned_addr learned my addr 172.21.15.88:0/914603780 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f31c2ffd640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c4155590 0x7f31c415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:32.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f320359e640 1 -- 172.21.15.88:0/2311165417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e8005080 con 0x7f31fc0df620 2024-09-21T02:03:32.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f31c37fe640 1 -- 172.21.15.88:0/914603780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c4155590 msgr2=0x7f31c415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f31c37fe640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c4155590 0x7f31c415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f31c37fe640 1 -- 172.21.15.88:0/914603780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c4150eb0 msgr2=0x7f31c415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f31c37fe640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c4150eb0 0x7f31c415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f31c37fe640 1 -- 172.21.15.88:0/914603780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c414ffa0 con 0x7f31c41f62f0 2024-09-21T02:03:32.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.910+0000 7f31c37fe640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c41f62f0 0x7f31c4150260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31b4004060 tx=0x7f31b4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:32.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.918+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f31f80055d0 con 0x7f31fc0df620 2024-09-21T02:03:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.924+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4001450 con 0x7f31c41f62f0 2024-09-21T02:03:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.924+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f31b40016c0 con 0x7f31c41f62f0 2024-09-21T02:03:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.924+0000 7f31ca132640 1 -- 172.21.15.88:0/914603780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c4151a90 con 0x7f31c41f62f0 2024-09-21T02:03:32.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.924+0000 7f31ca132640 1 -- 172.21.15.88:0/914603780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c4150990 con 0x7f31c41f62f0 2024-09-21T02:03:32.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.930+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4003850 con 0x7f31c41f62f0 2024-09-21T02:03:32.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.932+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31b4004a70 con 0x7f31c41f62f0 2024-09-21T02:03:32.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.933+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f31b403f420 con 0x7f31c41f62f0 2024-09-21T02:03:32.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.933+0000 7f31a9ffb640 1 -- 172.21.15.88:0/914603780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a4005080 con 0x7f31c41f62f0 2024-09-21T02:03:32.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.933+0000 7fd873f19640 1 -- 172.21.15.88:0/757198136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fd8480050e0 con 0x7fd86c0cd9e0 2024-09-21T02:03:32.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.942+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f31b4045060 con 0x7f31c41f62f0 2024-09-21T02:03:32.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.950+0000 7fd8627fc640 1 -- 172.21.15.88:0/757198136 <== mon.0 v2:172.21.15.88:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v6)=0 v6) ==== 108+0+0 (secure 0 0 0) 0x7fd868043a30 con 0x7fd86c0cd9e0 2024-09-21T02:03:32.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.967+0000 7fd873f19640 1 -- 172.21.15.88:0/757198136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd86c0cd9e0 msgr2=0x7fd86c13eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:32.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.967+0000 7fd873f19640 1 --2- 172.21.15.88:0/757198136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd86c0cd9e0 0x7fd86c13eca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd868009a40 tx=0x7fd8680046c0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.967+0000 7fd873f19640 1 -- 172.21.15.88:0/757198136 shutdown_connections 2024-09-21T02:03:32.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.967+0000 7fd873f19640 1 --2- 172.21.15.88:0/757198136 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd86c157ed0 0x7fd86c074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.968+0000 7fd873f19640 1 --2- 172.21.15.88:0/757198136 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd86c14e7d0 0x7fd86c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.968+0000 7fd873f19640 1 --2- 172.21.15.88:0/757198136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd86c0cd9e0 0x7fd86c13eca0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:32.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.968+0000 7fd873f19640 1 -- 172.21.15.88:0/757198136 >> 172.21.15.88:0/757198136 conn(0x7fd86c071570 msgr2=0x7fd86c0ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:32.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.971+0000 7fd873f19640 1 -- 172.21.15.88:0/757198136 shutdown_connections 2024-09-21T02:03:32.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:32.971+0000 7fd873f19640 1 -- 172.21.15.88:0/757198136 wait complete. 2024-09-21T02:03:33.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.018+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2370518926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f415c370 msgr2=0x7f64f4154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.018+0000 7f64fb7b0640 1 --2- 172.21.15.88:0/2370518926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f415c370 0x7f64f4154500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005fd0 tx=0x7f64ec02a840 comp rx=0 tx=0).stop 2024-09-21T02:03:33.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.019+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2370518926 shutdown_connections 2024-09-21T02:03:33.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.019+0000 7f64fb7b0640 1 --2- 172.21.15.88:0/2370518926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f415c370 0x7f64f4154500 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.019+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2370518926 >> 172.21.15.88:0/2370518926 conn(0x7f64f406d390 msgr2=0x7f64f40ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.022+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2370518926 shutdown_connections 2024-09-21T02:03:33.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.028+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2370518926 wait complete. 2024-09-21T02:03:33.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.031+0000 7f64fb7b0640 1 Processor -- start 2024-09-21T02:03:33.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.032+0000 7f64fb7b0640 1 -- start start 2024-09-21T02:03:33.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.034+0000 7f64fb7b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f4070540 0x7f64f42025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.034+0000 7f64fb7b0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f40b2f80 0x7f64f41fef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.035+0000 7f64f8d4b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f40b2f80 0x7f64f41fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.035+0000 7f64f8d4b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f40b2f80 0x7f64f41fef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41112/0 (socket says 172.21.15.88:41112) 2024-09-21T02:03:33.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.035+0000 7f64f8d4b640 1 -- 172.21.15.88:0/2777986799 learned_addr learned my addr 172.21.15.88:0/2777986799 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64fb7b0640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f415c370 0x7f64f40b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4072bf0 con 0x7f64f40b2f80 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4075a10 con 0x7f64f4070540 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f64f4070950 con 0x7f64f415c370 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64f8d4b640 1 -- 172.21.15.88:0/2777986799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f415c370 msgr2=0x7f64f40b1b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64f8d4b640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f415c370 0x7f64f40b1b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64f8d4b640 1 -- 172.21.15.88:0/2777986799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f4070540 msgr2=0x7f64f42025f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64f8d4b640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f4070540 0x7f64f42025f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.036+0000 7f64f8d4b640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f4202b30 con 0x7f64f40b2f80 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.037+0000 7f64f8d4b640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f40b2f80 0x7f64f41fef20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b7b0 tx=0x7f64e400bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.038+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4019070 con 0x7f64f40b2f80 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.039+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f64e400c810 con 0x7f64f40b2f80 2024-09-21T02:03:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.039+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f41ff870 con 0x7f64f40b2f80 2024-09-21T02:03:33.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.046+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f40ae8b0 con 0x7f64f40b2f80 2024-09-21T02:03:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.059+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e400cc40 con 0x7f64f40b2f80 2024-09-21T02:03:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.059+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f64e401c070 con 0x7f64f40b2f80 2024-09-21T02:03:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.059+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f64e401d880 con 0x7f64f40b2f80 2024-09-21T02:03:33.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.060+0000 7f64df7fe640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d4005080 con 0x7f64f40b2f80 2024-09-21T02:03:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.065+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f64e400cde0 con 0x7f64f40b2f80 2024-09-21T02:03:33.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.150+0000 7f54e67fc640 1 -- 172.21.15.88:0/2061340518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f54d4005670 con 0x7f54f80c28d0 2024-09-21T02:03:33.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.165+0000 7f54f4ff9640 1 -- 172.21.15.88:0/2061340518 <== mon.0 v2:172.21.15.88:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v7)=0 v7) ==== 117+0+0 (secure 0 0 0) 0x7f54ec03b6b0 con 0x7f54f80c28d0 2024-09-21T02:03:33.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.176+0000 7f54e67fc640 1 -- 172.21.15.88:0/2061340518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54f80c28d0 msgr2=0x7f54f8150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.176+0000 7f54e67fc640 1 --2- 172.21.15.88:0/2061340518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54f80c28d0 0x7f54f8150c50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f54ec031000 tx=0x7f54ec0046c0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.176+0000 7f54e67fc640 1 -- 172.21.15.88:0/2061340518 shutdown_connections 2024-09-21T02:03:33.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.177+0000 7f54e67fc640 1 --2- 172.21.15.88:0/2061340518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54f80c28d0 0x7f54f8150c50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.177+0000 7f54e67fc640 1 --2- 172.21.15.88:0/2061340518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54f80b1e60 0x7f54f8150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.177+0000 7f54e67fc640 1 --2- 172.21.15.88:0/2061340518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f80b18f0 0x7f54f8073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.177+0000 7f54e67fc640 1 -- 172.21.15.88:0/2061340518 >> 172.21.15.88:0/2061340518 conn(0x7f54f806d8f0 msgr2=0x7f54f8152940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.184+0000 7f54e67fc640 1 -- 172.21.15.88:0/2061340518 shutdown_connections 2024-09-21T02:03:33.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.184+0000 7f54e67fc640 1 -- 172.21.15.88:0/2061340518 wait complete. 2024-09-21T02:03:33.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.194+0000 7fbb80ff9640 1 -- 172.21.15.88:0/2875667756 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fbb78005370 con 0x7fbb940c97d0 2024-09-21T02:03:33.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.201+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31f8021440 con 0x7f31fc0df620 2024-09-21T02:03:33.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.204+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb88009dd0 con 0x7fbb940c97d0 2024-09-21T02:03:33.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.204+0000 7fbb82ffd640 1 -- 172.21.15.88:0/2875667756 <== mon.1 v2:172.21.15.195:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v8)=0 v8) ==== 123+0+0 (secure 0 0 0) 0x7fbb88010a10 con 0x7fbb940c97d0 2024-09-21T02:03:33.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.204+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64e401db70 con 0x7f64f40b2f80 2024-09-21T02:03:33.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.205+0000 7f137e7fc640 1 -- 172.21.15.88:0/3384395069 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c0047a0 con 0x7f13800c2550 2024-09-21T02:03:33.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.205+0000 7f1386207640 1 -- 172.21.15.88:0/3384395069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13800c2550 msgr2=0x7f13800c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.205+0000 7f1386207640 1 --2- 172.21.15.88:0/3384395069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13800c2550 0x7f13800c2960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f136c005fd0 tx=0x7f136c02a750 comp rx=0 tx=0).stop 2024-09-21T02:03:33.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.207+0000 7fbb9898a640 1 -- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb940c97d0 msgr2=0x7fbb9414ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.209+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31b40070d0 con 0x7f31c41f62f0 2024-09-21T02:03:33.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.209+0000 7f1386207640 1 -- 172.21.15.88:0/3384395069 shutdown_connections 2024-09-21T02:03:33.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.209+0000 7f1386207640 1 --2- 172.21.15.88:0/3384395069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13800c2550 0x7f13800c2960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.209+0000 7f1386207640 1 -- 172.21.15.88:0/3384395069 >> 172.21.15.88:0/3384395069 conn(0x7f138006cd60 msgr2=0x7f138006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.210+0000 7f1386207640 1 -- 172.21.15.88:0/3384395069 shutdown_connections 2024-09-21T02:03:33.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.214+0000 7f1386207640 1 -- 172.21.15.88:0/3384395069 wait complete. 2024-09-21T02:03:33.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.217+0000 7f1386207640 1 Processor -- start 2024-09-21T02:03:33.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.218+0000 7f1386207640 1 -- start start 2024-09-21T02:03:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb940c97d0 0x7fbb9414ca50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb880058c0 tx=0x7fbb8800db90 comp rx=0 tx=0).stop 2024-09-21T02:03:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 -- 172.21.15.88:0/2875667756 shutdown_connections 2024-09-21T02:03:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb94159d70 0x7fbb941e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb940c97d0 0x7fbb9414ca50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 --2- 172.21.15.88:0/2875667756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb940c72b0 0x7fbb94155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 -- 172.21.15.88:0/2875667756 >> 172.21.15.88:0/2875667756 conn(0x7fbb940720b0 msgr2=0x7fbb940cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 -- 172.21.15.88:0/2875667756 shutdown_connections 2024-09-21T02:03:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.224+0000 7fbb9898a640 1 -- 172.21.15.88:0/2875667756 wait complete. 2024-09-21T02:03:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.230+0000 7f1386207640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13800c2550 0x7f1380141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f1386207640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13801521a0 0x7f13801570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f808a47e640 1 -- 172.21.15.88:0/3655613715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f807c007190 msgr2=0x7f80740079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f808a47e640 1 --2- 172.21.15.88:0/3655613715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f807c007190 0x7f80740079a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f808406a560 tx=0x7f8074005830 comp rx=0 tx=0).stop 2024-09-21T02:03:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f1386207640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f138015b720 0x7f138013db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f1386207640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1380074ba0 con 0x7f138015b720 2024-09-21T02:03:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f1386207640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f13800748c0 con 0x7f13801521a0 2024-09-21T02:03:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f137ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f138015b720 0x7f138013db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f8088a1b640 1 -- 172.21.15.88:0/3655613715 reap_dead start 2024-09-21T02:03:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f808a47e640 1 -- 172.21.15.88:0/3655613715 shutdown_connections 2024-09-21T02:03:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f808a47e640 1 -- 172.21.15.88:0/3655613715 >> 172.21.15.88:0/3655613715 conn(0x7f807c04fe30 msgr2=0x7f807c050240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.241+0000 7f1275e07640 1 -- 172.21.15.88:0/1136552455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1270070540 msgr2=0x7f127016be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.241+0000 7f1275e07640 1 --2- 172.21.15.88:0/1136552455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1270070540 0x7f127016be80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12700c9460 tx=0x7f126002b200 comp rx=0 tx=0).stop 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.241+0000 7f1275e07640 1 -- 172.21.15.88:0/1136552455 shutdown_connections 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.241+0000 7f1275e07640 1 --2- 172.21.15.88:0/1136552455 >> v2:172.21.15.195:3300/0 conn(0x7f1270070950 0x7f127016c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.241+0000 7f1275e07640 1 --2- 172.21.15.88:0/1136552455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1270070540 0x7f127016be80 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12700c9460 tx=0x7f126002b200 comp rx=0 tx=0).stop 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.241+0000 7f1275e07640 1 --2- 172.21.15.88:0/1136552455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12700c7560 0x7f12700c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.241+0000 7f1275e07640 1 -- 172.21.15.88:0/1136552455 >> 172.21.15.88:0/1136552455 conn(0x7f127006d390 msgr2=0x7f12700ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.242+0000 7f1275e07640 1 -- 172.21.15.88:0/1136552455 shutdown_connections 2024-09-21T02:03:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.240+0000 7f808a47e640 1 -- 172.21.15.88:0/3655613715 shutdown_connections 2024-09-21T02:03:33.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.240+0000 7f808a47e640 1 -- 172.21.15.88:0/3655613715 wait complete. 2024-09-21T02:03:33.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.243+0000 7f808a47e640 1 Processor -- start 2024-09-21T02:03:33.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.243+0000 7f808a47e640 1 -- start start 2024-09-21T02:03:33.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.242+0000 7f1275e07640 1 -- 172.21.15.88:0/1136552455 wait complete. 2024-09-21T02:03:33.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.245+0000 7f808a47e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f807c0fd0e0 0x7f807c0fd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.245+0000 7f8083fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f807c0fd0e0 0x7f807c0fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.245+0000 7f8083fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f807c0fd0e0 0x7f807c0fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41156/0 (socket says 172.21.15.88:41156) 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.245+0000 7f8083fff640 1 -- 172.21.15.88:0/3455467713 learned_addr learned my addr 172.21.15.88:0/3455467713 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.245+0000 7f808a47e640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f807c0f7cf0 0x7f807c0fb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f808a47e640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f807c0f8100 0x7f807c0fb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f808a47e640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f807c04bba0 con 0x7f807c0fd0e0 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f808a47e640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f807c04b410 con 0x7f807c0f8100 2024-09-21T02:03:33.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f808a47e640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f807c04bd20 con 0x7f807c0f7cf0 2024-09-21T02:03:33.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f8083fff640 1 -- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f807c0f7cf0 msgr2=0x7f807c0fb300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f8083fff640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f807c0f7cf0 0x7f807c0fb300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f8083fff640 1 -- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f807c0f8100 msgr2=0x7f807c0fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f8083fff640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f807c0f8100 0x7f807c0fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.246+0000 7f8083fff640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c0ec5b0 con 0x7f807c0fd0e0 2024-09-21T02:03:33.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.247+0000 7f8083fff640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f807c0fd0e0 0x7f807c0fd970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8078006680 tx=0x7f80780066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f137ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f138015b720 0x7f138013db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41154/0 (socket says 172.21.15.88:41154) 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.239+0000 7f137ffff640 1 -- 172.21.15.88:0/1078432223 learned_addr learned my addr 172.21.15.88:0/1078432223 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.240+0000 7f137f7fe640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13800c2550 0x7f1380141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.240+0000 7f137effd640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13801521a0 0x7f13801570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.247+0000 7f8080ff9640 1 -- 172.21.15.88:0/3455467713 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078015070 con 0x7f807c0fd0e0 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.247+0000 7f8080ff9640 1 -- 172.21.15.88:0/3455467713 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f807800f980 con 0x7f807c0fd0e0 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.247+0000 7f8080ff9640 1 -- 172.21.15.88:0/3455467713 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807800fdd0 con 0x7f807c0fd0e0 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.247+0000 7f808a47e640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c0f85f0 con 0x7f807c0fd0e0 2024-09-21T02:03:33.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.248+0000 7f808a47e640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c0fa6c0 con 0x7f807c0fd0e0 2024-09-21T02:03:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.251+0000 7f1386207640 1 -- 172.21.15.88:0/1078432223 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f138006fe90 con 0x7f13800c2550 2024-09-21T02:03:33.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.251+0000 7f8080ff9640 1 -- 172.21.15.88:0/3455467713 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80780117c0 con 0x7f807c0fd0e0 2024-09-21T02:03:33.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.251+0000 7f8080ff9640 1 -- 172.21.15.88:0/3455467713 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8078019370 con 0x7f807c0fd0e0 2024-09-21T02:03:33.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.251+0000 7f806e7fc640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8064005290 con 0x7f807c0fd0e0 2024-09-21T02:03:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.252+0000 7f137effd640 1 -- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13800c2550 msgr2=0x7f1380141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.252+0000 7f137effd640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13800c2550 0x7f1380141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.252+0000 7f137effd640 1 -- 172.21.15.88:0/1078432223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f138015b720 msgr2=0x7f138013db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.252+0000 7f137effd640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f138015b720 0x7f138013db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.252+0000 7f137effd640 1 -- 172.21.15.88:0/1078432223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1380142310 con 0x7f13801521a0 2024-09-21T02:03:33.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.254+0000 7f137effd640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13801521a0 0x7f13801570a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f138006f940 tx=0x7f136c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 -- 172.21.15.88:0/2668583426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 msgr2=0x7f366c16c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 --2- 172.21.15.88:0/2668583426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 0x7f366c16c590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3664005fd0 tx=0x7f366402e720 comp rx=0 tx=0).stop 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 -- 172.21.15.88:0/2668583426 shutdown_connections 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 --2- 172.21.15.88:0/2668583426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 0x7f366c16c590 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 --2- 172.21.15.88:0/2668583426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f366c0c7560 0x7f366c0c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 -- 172.21.15.88:0/2668583426 >> 172.21.15.88:0/2668583426 conn(0x7f366c06d390 msgr2=0x7f366c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 -- 172.21.15.88:0/2668583426 shutdown_connections 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.254+0000 7f136bfff640 1 -- 172.21.15.88:0/1078432223 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c03e030 con 0x7f13801521a0 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.254+0000 7f136bfff640 1 -- 172.21.15.88:0/1078432223 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f136c0047a0 con 0x7f13801521a0 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.254+0000 7f1386207640 1 -- 172.21.15.88:0/1078432223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138015a350 con 0x7f13801521a0 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.254+0000 7f1386207640 1 -- 172.21.15.88:0/1078432223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138015a910 con 0x7f13801521a0 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.256+0000 7f136bfff640 1 -- 172.21.15.88:0/1078432223 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c035040 con 0x7f13801521a0 2024-09-21T02:03:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.256+0000 7f136bfff640 1 -- 172.21.15.88:0/1078432223 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f136c041030 con 0x7f13801521a0 2024-09-21T02:03:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.256+0000 7f136bfff640 1 -- 172.21.15.88:0/1078432223 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f136c031740 con 0x7f13801521a0 2024-09-21T02:03:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.256+0000 7f8080ff9640 1 -- 172.21.15.88:0/3455467713 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8078011c90 con 0x7f807c0fd0e0 2024-09-21T02:03:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.258+0000 7f1369ffb640 1 -- 172.21.15.88:0/1078432223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1360005290 con 0x7f13801521a0 2024-09-21T02:03:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.255+0000 7f367084d640 1 -- 172.21.15.88:0/2668583426 wait complete. 2024-09-21T02:03:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.258+0000 7f367084d640 1 Processor -- start 2024-09-21T02:03:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.258+0000 7f367084d640 1 -- start start 2024-09-21T02:03:33.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7f6db11eb640 1 -- 172.21.15.88:0/1949479433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac070540 msgr2=0x7f6dac074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7f6db11eb640 1 --2- 172.21.15.88:0/1949479433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac070540 0x7f6dac074320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6dac08def0 tx=0x7f6d9c02a560 comp rx=0 tx=0).stop 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.263+0000 7f6db11eb640 1 -- 172.21.15.88:0/1949479433 shutdown_connections 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.263+0000 7f6db11eb640 1 --2- 172.21.15.88:0/1949479433 >> v2:172.21.15.195:3300/0 conn(0x7f6dac070950 0x7f6dac074860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.263+0000 7f6db11eb640 1 --2- 172.21.15.88:0/1949479433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac070540 0x7f6dac074320 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.263+0000 7f6db11eb640 1 -- 172.21.15.88:0/1949479433 >> 172.21.15.88:0/1949479433 conn(0x7f6dac06d390 msgr2=0x7f6dac0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7fd4cce25640 1 -- 172.21.15.88:0/2778311772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8070540 msgr2=0x7fd4c815bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7fd4cce25640 1 --2- 172.21.15.88:0/2778311772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8070540 0x7fd4c815bc60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005fd0 tx=0x7fd4bc02a560 comp rx=0 tx=0).stop 2024-09-21T02:03:33.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7f136bfff640 1 -- 172.21.15.88:0/1078432223 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f136c0094d0 con 0x7f13801521a0 2024-09-21T02:03:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7fa1672b4640 1 -- 172.21.15.88:0/1401140111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa14c005350 msgr2=0x7fa14c00b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7fa1672b4640 1 --2- 172.21.15.88:0/1401140111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa14c005350 0x7fa14c00b9e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c610 tx=0x7fa14c005830 comp rx=0 tx=0).stop 2024-09-21T02:03:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7fa165851640 1 -- 172.21.15.88:0/1401140111 reap_dead start 2024-09-21T02:03:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7fa1672b4640 1 -- 172.21.15.88:0/1401140111 shutdown_connections 2024-09-21T02:03:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.262+0000 7fa1672b4640 1 -- 172.21.15.88:0/1401140111 >> 172.21.15.88:0/1401140111 conn(0x7fa16006d390 msgr2=0x7fa1600ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.244+0000 7f1275e07640 1 Processor -- start 2024-09-21T02:03:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.266+0000 7f1275e07640 1 -- start start 2024-09-21T02:03:33.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.263+0000 7f6db11eb640 1 -- 172.21.15.88:0/1949479433 shutdown_connections 2024-09-21T02:03:33.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.263+0000 7f6db11eb640 1 -- 172.21.15.88:0/1949479433 wait complete. 2024-09-21T02:03:33.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.267+0000 7f6db11eb640 1 Processor -- start 2024-09-21T02:03:33.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.267+0000 7fd4cce25640 1 -- 172.21.15.88:0/2778311772 shutdown_connections 2024-09-21T02:03:33.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.267+0000 7fd4cce25640 1 --2- 172.21.15.88:0/2778311772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8070540 0x7fd4c815bc60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.267+0000 7fd4cce25640 1 --2- 172.21.15.88:0/2778311772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4c80b2f80 0x7fd4c80750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.267+0000 7fd4cce25640 1 -- 172.21.15.88:0/2778311772 >> 172.21.15.88:0/2778311772 conn(0x7fd4c806d390 msgr2=0x7fd4c80ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.268+0000 7f1275e07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1270070950 0x7f12702029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.268+0000 7f1275e07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12700c7560 0x7f12701ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.268+0000 7f1275e07640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12700c6810 0x7f127015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.268+0000 7f1275e07640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12700cd8e0 con 0x7f12700c6810 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.268+0000 7f1275e07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12700c21d0 con 0x7f12700c7560 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f1275e07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12700c3c30 con 0x7f1270070950 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12700c6810 0x7f127015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12700c6810 0x7f127015d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41178/0 (socket says 172.21.15.88:41178) 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 -- 172.21.15.88:0/849926326 learned_addr learned my addr 172.21.15.88:0/849926326 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 -- 172.21.15.88:0/849926326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1270070950 msgr2=0x7f12702029b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 --2- 172.21.15.88:0/849926326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1270070950 0x7f12702029b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 -- 172.21.15.88:0/849926326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12700c7560 msgr2=0x7f12701ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 --2- 172.21.15.88:0/849926326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12700c7560 0x7f12701ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.269+0000 7f126ffff640 1 -- 172.21.15.88:0/849926326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12701fea00 con 0x7f12700c6810 2024-09-21T02:03:33.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.270+0000 7f126ffff640 1 --2- 172.21.15.88:0/849926326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12700c6810 0x7f127015d0b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f126400d990 tx=0x7f126400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.270+0000 7fa1672b4640 1 -- 172.21.15.88:0/1401140111 shutdown_connections 2024-09-21T02:03:33.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f367084d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f366c070540 0x7f366c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f367084d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 0x7f366c0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f366a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f366c070540 0x7f366c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f366a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f366c070540 0x7f366c159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41194/0 (socket says 172.21.15.88:41194) 2024-09-21T02:03:33.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.271+0000 7f1257fff640 1 -- 172.21.15.88:0/849926326 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126401b070 con 0x7f12700c6810 2024-09-21T02:03:33.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.271+0000 7f1257fff640 1 -- 172.21.15.88:0/849926326 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f126400ed50 con 0x7f12700c6810 2024-09-21T02:03:33.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.271+0000 7f1257fff640 1 -- 172.21.15.88:0/849926326 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12640166a0 con 0x7f12700c6810 2024-09-21T02:03:33.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.271+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12700ccb00 con 0x7f12700c6810 2024-09-21T02:03:33.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.271+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12701fecc0 con 0x7f12700c6810 2024-09-21T02:03:33.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f366a59c640 1 -- 172.21.15.88:0/156714292 learned_addr learned my addr 172.21.15.88:0/156714292 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f367084d640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f366c0c7560 0x7f366c0c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f367084d640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0c21d0 con 0x7f366c070540 2024-09-21T02:03:33.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f367084d640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0c1ef0 con 0x7f366c070950 2024-09-21T02:03:33.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f367084d640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f366c0cd8e0 con 0x7f366c0c7560 2024-09-21T02:03:33.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.272+0000 7f366ad9d640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f366c0c7560 0x7f366c0c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.267+0000 7f6db11eb640 1 -- start start 2024-09-21T02:03:33.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.273+0000 7f1257fff640 1 -- 172.21.15.88:0/849926326 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f126401e070 con 0x7f12700c6810 2024-09-21T02:03:33.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.273+0000 7f1257fff640 1 -- 172.21.15.88:0/849926326 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1264020db0 con 0x7f12700c6810 2024-09-21T02:03:33.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.273+0000 7fa1672b4640 1 -- 172.21.15.88:0/1401140111 wait complete. 2024-09-21T02:03:33.276 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.276 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.277 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.277 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.274+0000 7f6db11eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dac070950 0x7f6dac1692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.274+0000 7f6db11eb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac0c7560 0x7f6dac0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f6daa59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac0c7560 0x7f6dac0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7fd4cce25640 1 -- 172.21.15.88:0/2778311772 shutdown_connections 2024-09-21T02:03:33.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f6daa59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac0c7560 0x7f6dac0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41198/0 (socket says 172.21.15.88:41198) 2024-09-21T02:03:33.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f6daa59c640 1 -- 172.21.15.88:0/2709451139 learned_addr learned my addr 172.21.15.88:0/2709451139 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f6db11eb640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dac16a1b0 0x7f6dac15b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac0cd8e0 con 0x7f6dac0c7560 2024-09-21T02:03:33.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac0c21d0 con 0x7f6dac070950 2024-09-21T02:03:33.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6dac0c3c30 con 0x7f6dac16a1b0 2024-09-21T02:03:33.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.276+0000 7fa1672b4640 1 Processor -- start 2024-09-21T02:03:33.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.276+0000 7fa1672b4640 1 -- start start 2024-09-21T02:03:33.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.276+0000 7fd4cce25640 1 -- 172.21.15.88:0/2778311772 wait complete. 2024-09-21T02:03:33.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.279+0000 7fd4cce25640 1 Processor -- start 2024-09-21T02:03:33.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.279+0000 7fd4cce25640 1 -- start start 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.281+0000 7fd4cce25640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4c8070540 0x7fd4c815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.281+0000 7fd4cce25640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4c8070950 0x7fd4c81f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fd4cce25640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80b2f80 0x7fd4c80c2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fd4cce25640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8072bf0 con 0x7fd4c80b2f80 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fd4cce25640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8075a10 con 0x7fd4c8070950 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fd4cce25640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c8075630 con 0x7fd4c8070540 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.278+0000 7fa1672b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa16015d1a0 0x7fa160158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.278+0000 7fa1672b4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1601536a0 0x7fa160153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.278+0000 7fa1672b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1601f9b70 0x7fa1601f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.278+0000 7fa1672b4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa160072bf0 con 0x7fa1601536a0 2024-09-21T02:03:33.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.278+0000 7fa1672b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa160075a10 con 0x7fa16015d1a0 2024-09-21T02:03:33.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.278+0000 7fa1672b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa16015c1a0 con 0x7fa1601f9b70 2024-09-21T02:03:33.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.276+0000 7f6daad9d640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dac070950 0x7f6dac1692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.276+0000 7f6dab59e640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dac16a1b0 0x7f6dac15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.279+0000 7f6daa59c640 1 -- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dac16a1b0 msgr2=0x7f6dac15b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.279+0000 7f6daa59c640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dac16a1b0 0x7f6dac15b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fd4c6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80b2f80 0x7fd4c80c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.279+0000 7f6daa59c640 1 -- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dac070950 msgr2=0x7f6dac1692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fd4c6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80b2f80 0x7fd4c80c2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41208/0 (socket says 172.21.15.88:41208) 2024-09-21T02:03:33.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fd4c6d9d640 1 -- 172.21.15.88:0/229097348 learned_addr learned my addr 172.21.15.88:0/229097348 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.279+0000 7f6daa59c640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dac070950 0x7f6dac1692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.279+0000 7f6daa59c640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac0ccd20 con 0x7f6dac0c7560 2024-09-21T02:03:33.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.280+0000 7f6daa59c640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac0c7560 0x7f6dac0cc7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009920 tx=0x7f6d9c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7f6d937fe640 1 -- 172.21.15.88:0/2709451139 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c041070 con 0x7f6dac0c7560 2024-09-21T02:03:33.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fa16484f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1601536a0 0x7fa160153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fa16484f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1601536a0 0x7fa160153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41202/0 (socket says 172.21.15.88:41202) 2024-09-21T02:03:33.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.282+0000 7fa16484f640 1 -- 172.21.15.88:0/1601429365 learned_addr learned my addr 172.21.15.88:0/1601429365 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fa165050640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa16015d1a0 0x7fa160158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7f6d937fe640 1 -- 172.21.15.88:0/2709451139 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d9c003e90 con 0x7f6dac0c7560 2024-09-21T02:03:33.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac1fee00 con 0x7f6dac0c7560 2024-09-21T02:03:33.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac0cbfe0 con 0x7f6dac0c7560 2024-09-21T02:03:33.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.275+0000 7f3669d9b640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 0x7f366c0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fa16484f640 1 -- 172.21.15.88:0/1601429365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1601f9b70 msgr2=0x7fa1601f9f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:03:33.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fa16484f640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1601f9b70 0x7fa1601f9f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fa16484f640 1 -- 172.21.15.88:0/1601429365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa16015d1a0 msgr2=0x7fa160158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fa16484f640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa16015d1a0 0x7fa160158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fa16484f640 1 -- 172.21.15.88:0/1601429365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1601544e0 con 0x7fa1601536a0 2024-09-21T02:03:33.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.284+0000 7f366a59c640 1 -- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f366c0c7560 msgr2=0x7f366c0c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.284+0000 7f366a59c640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f366c0c7560 0x7f366c0c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.284+0000 7f366a59c640 1 -- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 msgr2=0x7f366c0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.284+0000 7f366a59c640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 0x7f366c0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.284+0000 7f366a59c640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c0c56e0 con 0x7f366c070540 2024-09-21T02:03:33.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.284+0000 7f366a59c640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f366c070540 0x7f366c159b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f365c007890 tx=0x7f365c007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.285+0000 7f6d937fe640 1 -- 172.21.15.88:0/2709451139 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c032e60 con 0x7f6dac0c7560 2024-09-21T02:03:33.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.285+0000 7f365affd640 1 -- 172.21.15.88:0/156714292 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c019020 con 0x7f366c070540 2024-09-21T02:03:33.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.285+0000 7f365affd640 1 -- 172.21.15.88:0/156714292 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f365c00fc10 con 0x7f366c070540 2024-09-21T02:03:33.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.285+0000 7f365affd640 1 -- 172.21.15.88:0/156714292 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c015470 con 0x7f366c070540 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.285+0000 7f6d937fe640 1 -- 172.21.15.88:0/2709451139 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6d9c044070 con 0x7f6dac0c7560 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.285+0000 7f6d937fe640 1 -- 172.21.15.88:0/2709451139 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6d9c046c20 con 0x7f6dac0c7560 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.287+0000 7f6d917fa640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c005290 con 0x7f6dac0c7560 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.287+0000 7fd4c6d9d640 1 -- 172.21.15.88:0/229097348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4c8070540 msgr2=0x7fd4c815f0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.287+0000 7fd4c6d9d640 1 --2- 172.21.15.88:0/229097348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4c8070540 0x7fd4c815f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.287+0000 7fd4c6d9d640 1 -- 172.21.15.88:0/229097348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4c8070950 msgr2=0x7fd4c81f6e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.287+0000 7fd4c6d9d640 1 --2- 172.21.15.88:0/229097348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4c8070950 0x7fd4c81f6e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.287+0000 7fd4c6d9d640 1 -- 172.21.15.88:0/229097348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c814ee90 con 0x7fd4c80b2f80 2024-09-21T02:03:33.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.288+0000 7fd4c6d9d640 1 --2- 172.21.15.88:0/229097348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80b2f80 0x7fd4c80c2030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000b9e0 tx=0x7fd4c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.288+0000 7fa16484f640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1601536a0 0x7fa160153ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b390 tx=0x7fa15000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.291+0000 7fd4b6ffd640 1 -- 172.21.15.88:0/229097348 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0019070 con 0x7fd4c80b2f80 2024-09-21T02:03:33.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.291+0000 7fd4b6ffd640 1 -- 172.21.15.88:0/229097348 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4c000cc10 con 0x7fd4c80b2f80 2024-09-21T02:03:33.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.291+0000 7fd4cce25640 1 -- 172.21.15.88:0/229097348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c81f6360 con 0x7fd4c80b2f80 2024-09-21T02:03:33.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fca90d20640 1 -- 172.21.15.88:0/3462485392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070540 msgr2=0x7fca8c15bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fca90d20640 1 --2- 172.21.15.88:0/3462485392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070540 0x7fca8c15bc60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca84005fd0 tx=0x7fca8402a9f0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fca90d20640 1 -- 172.21.15.88:0/3462485392 shutdown_connections 2024-09-21T02:03:33.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fca90d20640 1 --2- 172.21.15.88:0/3462485392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070540 0x7fca8c15bc60 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca84005fd0 tx=0x7fca8402a9f0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fca90d20640 1 --2- 172.21.15.88:0/3462485392 >> v2:172.21.15.195:3300/0 conn(0x7fca8c0b2f80 0x7fca8c0750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.283+0000 7fca90d20640 1 -- 172.21.15.88:0/3462485392 >> 172.21.15.88:0/3462485392 conn(0x7fca8c06d390 msgr2=0x7fca8c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.292+0000 7fd4cce25640 1 -- 172.21.15.88:0/229097348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c814f430 con 0x7fd4c80b2f80 2024-09-21T02:03:33.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.292+0000 7fca90d20640 1 -- 172.21.15.88:0/3462485392 shutdown_connections 2024-09-21T02:03:33.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.293+0000 7fd4b6ffd640 1 -- 172.21.15.88:0/229097348 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0014440 con 0x7fd4c80b2f80 2024-09-21T02:03:33.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.293+0000 7f22fc350640 1 -- 172.21.15.88:0/3025317320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4152840 msgr2=0x7f22f4074690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.293+0000 7f22fc350640 1 --2- 172.21.15.88:0/3025317320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4152840 0x7f22f4074690 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009860 tx=0x7f22f002a5f0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7f22fc350640 1 -- 172.21.15.88:0/3025317320 shutdown_connections 2024-09-21T02:03:33.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7f22fc350640 1 --2- 172.21.15.88:0/3025317320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f415c510 0x7f22f4162fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7f22fc350640 1 --2- 172.21.15.88:0/3025317320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4152840 0x7f22f4074690 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7f22fc350640 1 -- 172.21.15.88:0/3025317320 >> 172.21.15.88:0/3025317320 conn(0x7f22f406d8f0 msgr2=0x7f22f40ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7fd4b6ffd640 1 -- 172.21.15.88:0/229097348 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4c001c070 con 0x7fd4c80b2f80 2024-09-21T02:03:33.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7f22fc350640 1 -- 172.21.15.88:0/3025317320 shutdown_connections 2024-09-21T02:03:33.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7fd4b6ffd640 1 -- 172.21.15.88:0/229097348 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4c001e990 con 0x7fd4c80b2f80 2024-09-21T02:03:33.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.292+0000 7fca90d20640 1 -- 172.21.15.88:0/3462485392 wait complete. 2024-09-21T02:03:33.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.295+0000 7fca90d20640 1 Processor -- start 2024-09-21T02:03:33.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.295+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1250005290 con 0x7f12700c6810 2024-09-21T02:03:33.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.295+0000 7fd4b4ff9640 1 -- 172.21.15.88:0/229097348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ac005290 con 0x7fd4c80b2f80 2024-09-21T02:03:33.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.295+0000 7fca90d20640 1 -- start start 2024-09-21T02:03:33.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.296+0000 7f6d937fe640 1 -- 172.21.15.88:0/2709451139 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6d9c046450 con 0x7f6dac0c7560 2024-09-21T02:03:33.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.296+0000 7fa155ffb640 1 -- 172.21.15.88:0/1601429365 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150015020 con 0x7fa1601536a0 2024-09-21T02:03:33.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.297+0000 7fa155ffb640 1 -- 172.21.15.88:0/1601429365 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa150011940 con 0x7fa1601536a0 2024-09-21T02:03:33.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.297+0000 7fa1672b4640 1 -- 172.21.15.88:0/1601429365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1601f72b0 con 0x7fa1601536a0 2024-09-21T02:03:33.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.294+0000 7f22fc350640 1 -- 172.21.15.88:0/3025317320 wait complete. 2024-09-21T02:03:33.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.297+0000 7f22fc350640 1 Processor -- start 2024-09-21T02:03:33.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.297+0000 7fa1672b4640 1 -- 172.21.15.88:0/1601429365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa160152b10 con 0x7fa1601536a0 2024-09-21T02:03:33.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.297+0000 7f22fc350640 1 -- start start 2024-09-21T02:03:33.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.298+0000 7fa155ffb640 1 -- 172.21.15.88:0/1601429365 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150011d90 con 0x7fa1601536a0 2024-09-21T02:03:33.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.298+0000 7f367084d640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c172590 con 0x7f366c070540 2024-09-21T02:03:33.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.298+0000 7f367084d640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c15bc10 con 0x7f366c070540 2024-09-21T02:03:33.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22fc350640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4074bd0 0x7f22f40c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22fc350640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f4152840 0x7f22f41f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22fc350640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f415c510 0x7f22f415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22fc350640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4152c50 con 0x7f22f4152840 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22f98eb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f4152840 0x7f22f41f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22f98eb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f4152840 0x7f22f41f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41210/0 (socket says 172.21.15.88:41210) 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22f98eb640 1 -- 172.21.15.88:0/1690461570 learned_addr learned my addr 172.21.15.88:0/1690461570 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.299+0000 7f22fc350640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4075990 con 0x7f22f4074bd0 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7f22fc350640 1 -- 172.21.15.88:0/1690461570 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f22f4152dd0 con 0x7f22f415c510 2024-09-21T02:03:33.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7f3658ff9640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c0cbc00 con 0x7f366c070540 2024-09-21T02:03:33.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7fa14b7fe640 1 -- 172.21.15.88:0/1601429365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa140005290 con 0x7fa1601536a0 2024-09-21T02:03:33.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7f22fa8ed640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f415c510 0x7f22f415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7f22fa0ec640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4074bd0 0x7f22f40c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7fca90d20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070950 0x7fca8c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7fca90d20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b2f80 0x7fca8c14e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.300+0000 7fca90d20640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0c6d00 0x7fca8c149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7fca90d20640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c072bf0 con 0x7fca8c0c6d00 2024-09-21T02:03:33.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7fca90d20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c075a10 con 0x7fca8c0b2f80 2024-09-21T02:03:33.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7fca90d20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fca8c075630 con 0x7fca8c070950 2024-09-21T02:03:33.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7f22f98eb640 1 -- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f415c510 msgr2=0x7f22f415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7f22f98eb640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f415c510 0x7f22f415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7f22f98eb640 1 -- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4074bd0 msgr2=0x7f22f40c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7f22f98eb640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4074bd0 0x7f22f40c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7f22f98eb640 1 -- 172.21.15.88:0/1690461570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f40c6860 con 0x7f22f4152840 2024-09-21T02:03:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.301+0000 7f22f98eb640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f4152840 0x7f22f41f5fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c430 tx=0x7f22e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.302+0000 7fca8a59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070950 0x7fca8c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.302+0000 7fca8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0c6d00 0x7fca8c149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.302+0000 7fca8ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0c6d00 0x7fca8c149460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41218/0 (socket says 172.21.15.88:41218) 2024-09-21T02:03:33.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.302+0000 7fca8ad9d640 1 -- 172.21.15.88:0/371328589 learned_addr learned my addr 172.21.15.88:0/371328589 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.302+0000 7fca89d9b640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b2f80 0x7fca8c14e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.306+0000 7fca8a59c640 1 -- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b2f80 msgr2=0x7fca8c14e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.306+0000 7fca8a59c640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b2f80 0x7fca8c14e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.306+0000 7fca8a59c640 1 -- 172.21.15.88:0/371328589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0c6d00 msgr2=0x7fca8c149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.306+0000 7fca8a59c640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0c6d00 0x7fca8c149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.306+0000 7fca8a59c640 1 -- 172.21.15.88:0/371328589 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c0cc750 con 0x7fca8c070950 2024-09-21T02:03:33.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.314+0000 7fca8a59c640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070950 0x7fca8c0cbe50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00bfd0 tx=0x7fca7c00da70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.314+0000 7fca7affd640 1 -- 172.21.15.88:0/371328589 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c017070 con 0x7fca8c070950 2024-09-21T02:03:33.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.314+0000 7fca7affd640 1 -- 172.21.15.88:0/371328589 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca7c0099e0 con 0x7fca8c070950 2024-09-21T02:03:33.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.314+0000 7fca7affd640 1 -- 172.21.15.88:0/371328589 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c009df0 con 0x7fca8c070950 2024-09-21T02:03:33.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.315+0000 7f3113a7e640 1 -- 172.21.15.88:0/26357831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c0b2f80 msgr2=0x7f310c0750f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.315+0000 7f3113a7e640 1 --2- 172.21.15.88:0/26357831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c0b2f80 0x7f310c0750f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3104005fd0 tx=0x7f310402a960 comp rx=0 tx=0).stop 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.315+0000 7f3113a7e640 1 -- 172.21.15.88:0/26357831 shutdown_connections 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.315+0000 7f3113a7e640 1 --2- 172.21.15.88:0/26357831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c0b2f80 0x7f310c0750f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.315+0000 7f3113a7e640 1 -- 172.21.15.88:0/26357831 >> 172.21.15.88:0/26357831 conn(0x7f310c06d390 msgr2=0x7f310c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.316+0000 7fca90d20640 1 -- 172.21.15.88:0/371328589 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c0c59a0 con 0x7fca8c070950 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.316+0000 7fca90d20640 1 -- 172.21.15.88:0/371328589 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c0c7250 con 0x7fca8c070950 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.316+0000 7fca7affd640 1 -- 172.21.15.88:0/371328589 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca7c01a070 con 0x7fca8c070950 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.316+0000 7fca7affd640 1 -- 172.21.15.88:0/371328589 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca7c01ca10 con 0x7fca8c070950 2024-09-21T02:03:33.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.317+0000 7f3113a7e640 1 -- 172.21.15.88:0/26357831 shutdown_connections 2024-09-21T02:03:33.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.317+0000 7fca78ff9640 1 -- 172.21.15.88:0/371328589 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca64005290 con 0x7fca8c070950 2024-09-21T02:03:33.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.321+0000 7f365affd640 1 -- 172.21.15.88:0/156714292 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f365c01c020 con 0x7f366c070540 2024-09-21T02:03:33.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.321+0000 7f365affd640 1 -- 172.21.15.88:0/156714292 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f365c028a60 con 0x7f366c070540 2024-09-21T02:03:33.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.321+0000 7f365affd640 1 -- 172.21.15.88:0/156714292 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f365c023020 con 0x7f366c070540 2024-09-21T02:03:33.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.321+0000 7f1257fff640 1 -- 172.21.15.88:0/849926326 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1264025070 con 0x7f12700c6810 2024-09-21T02:03:33.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.321+0000 7fd4b6ffd640 1 -- 172.21.15.88:0/229097348 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd4c001e440 con 0x7fd4c80b2f80 2024-09-21T02:03:33.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.317+0000 7f3113a7e640 1 -- 172.21.15.88:0/26357831 wait complete. 2024-09-21T02:03:33.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.320+0000 7f3113a7e640 1 Processor -- start 2024-09-21T02:03:33.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.320+0000 7f3113a7e640 1 -- start start 2024-09-21T02:03:33.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.321+0000 7fa155ffb640 1 -- 172.21.15.88:0/1601429365 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa150018020 con 0x7fa1601536a0 2024-09-21T02:03:33.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.321+0000 7fa155ffb640 1 -- 172.21.15.88:0/1601429365 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa15001a810 con 0x7fa1601536a0 2024-09-21T02:03:33.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.320+0000 7fca7affd640 1 -- 172.21.15.88:0/371328589 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fca7c010560 con 0x7fca8c070950 2024-09-21T02:03:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.325+0000 7f22eaffd640 1 -- 172.21.15.88:0/1690461570 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4011850 con 0x7f22f4152840 2024-09-21T02:03:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.325+0000 7f22eaffd640 1 -- 172.21.15.88:0/1690461570 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22e4017070 con 0x7f22f4152840 2024-09-21T02:03:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.325+0000 7f22eaffd640 1 -- 172.21.15.88:0/1690461570 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4011d40 con 0x7f22f4152840 2024-09-21T02:03:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.326+0000 7f22fc350640 1 -- 172.21.15.88:0/1690461570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f40c6f50 con 0x7f22f4152840 2024-09-21T02:03:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.326+0000 7f22fc350640 1 -- 172.21.15.88:0/1690461570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f4153350 con 0x7f22f4152840 2024-09-21T02:03:33.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.327+0000 7f3113a7e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310c15c370 0x7f310c0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.327+0000 7f2802e97640 1 -- 172.21.15.88:0/2638492229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27fc0cd000 msgr2=0x7f27fc15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.327+0000 7f2802e97640 1 --2- 172.21.15.88:0/2638492229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27fc0cd000 0x7f27fc15bc90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0c86e0 tx=0x7f27ec02a460 comp rx=0 tx=0).stop 2024-09-21T02:03:33.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.327+0000 7f2802e97640 1 -- 172.21.15.88:0/2638492229 shutdown_connections 2024-09-21T02:03:33.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.327+0000 7f2802e97640 1 --2- 172.21.15.88:0/2638492229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27fc0cd000 0x7f27fc15bc90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.327+0000 7f2802e97640 1 --2- 172.21.15.88:0/2638492229 >> v2:172.21.15.88:3300/0 conn(0x7f27fc072bf0 0x7f27fc073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.327+0000 7f2802e97640 1 -- 172.21.15.88:0/2638492229 >> 172.21.15.88:0/2638492229 conn(0x7f27fc06cf70 msgr2=0x7f27fc06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f22e8ff9640 1 -- 172.21.15.88:0/1690461570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f40c9180 con 0x7f22f4152840 2024-09-21T02:03:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f311181a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310c15c370 0x7f310c0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f311181a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310c15c370 0x7f310c0c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41234/0 (socket says 172.21.15.88:41234) 2024-09-21T02:03:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f311181a640 1 -- 172.21.15.88:0/3174784991 learned_addr learned my addr 172.21.15.88:0/3174784991 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f3113a7e640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f310c153310 0x7f310c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f3113a7e640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c153e30 0x7f310c14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f3111019640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f310c153310 0x7f310c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f310c072bf0 con 0x7f310c15c370 2024-09-21T02:03:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.332+0000 7f311201b640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c153e30 0x7f310c14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f056995e640 1 -- 172.21.15.88:0/1092339735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f054c003aa0 msgr2=0x7f054c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f056995e640 1 --2- 172.21.15.88:0/1092339735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f054c003aa0 0x7f054c003f10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f055c001e60 tx=0x7f054c005830 comp rx=0 tx=0).stop 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f310c075a10 con 0x7f310c153e30 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f310c075630 con 0x7f310c153310 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.337+0000 7f311181a640 1 -- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f310c153310 msgr2=0x7f310c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.337+0000 7f311181a640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f310c153310 0x7f310c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.337+0000 7f311181a640 1 -- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c153e30 msgr2=0x7f310c14f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.337+0000 7f311181a640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c153e30 0x7f310c14f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.337+0000 7f311181a640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c1fecd0 con 0x7f310c15c370 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.337+0000 7f311181a640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310c15c370 0x7f310c0c6860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f310402af50 tx=0x7f3104003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f056995e640 1 -- 172.21.15.88:0/1092339735 shutdown_connections 2024-09-21T02:03:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f056995e640 1 --2- 172.21.15.88:0/1092339735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f054c003aa0 0x7f054c003f10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f056995e640 1 -- 172.21.15.88:0/1092339735 >> 172.21.15.88:0/1092339735 conn(0x7f056406d390 msgr2=0x7f05640ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f056995e640 1 -- 172.21.15.88:0/1092339735 shutdown_connections 2024-09-21T02:03:33.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.338+0000 7f2802e97640 1 -- 172.21.15.88:0/2638492229 shutdown_connections 2024-09-21T02:03:33.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.336+0000 7f056995e640 1 -- 172.21.15.88:0/1092339735 wait complete. 2024-09-21T02:03:33.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.339+0000 7f056995e640 1 Processor -- start 2024-09-21T02:03:33.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.339+0000 7f056995e640 1 -- start start 2024-09-21T02:03:33.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.342+0000 7f2802e97640 1 -- 172.21.15.88:0/2638492229 wait complete. 2024-09-21T02:03:33.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.343+0000 7f056995e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05640c69b0 0x7f0564163710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.344+0000 7f2802e97640 1 Processor -- start 2024-09-21T02:03:33.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.344+0000 7f2802e97640 1 -- start start 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2802e97640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27fc072bf0 0x7f27fc1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2802e97640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27fc0cd000 0x7f27fc153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2802e97640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27fc0cd410 0x7f27fc0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27fc0cd000 0x7f27fc153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2802e97640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc0c8740 con 0x7f27fc0cd000 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2802e97640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc073e80 con 0x7f27fc0cd410 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2802e97640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc0c88c0 con 0x7f27fc072bf0 2024-09-21T02:03:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27fc0cd000 0x7f27fc153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41260/0 (socket says 172.21.15.88:41260) 2024-09-21T02:03:33.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 -- 172.21.15.88:0/1054366902 learned_addr learned my addr 172.21.15.88:0/1054366902 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 -- 172.21.15.88:0/1054366902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27fc072bf0 msgr2=0x7f27fc1536b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:03:33.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 --2- 172.21.15.88:0/1054366902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27fc072bf0 0x7f27fc1536b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 -- 172.21.15.88:0/1054366902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27fc0cd410 msgr2=0x7f27fc0c5630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 --2- 172.21.15.88:0/1054366902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27fc0cd410 0x7f27fc0c5630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.347+0000 7f2801694640 1 -- 172.21.15.88:0/1054366902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc1f79f0 con 0x7f27fc0cd000 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.343+0000 7f056995e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05640c5cb0 0x7f05640cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.343+0000 7f056995e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f056415e4d0 0x7f056415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.343+0000 7f056995e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05640cd8e0 con 0x7f05640c5cb0 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.343+0000 7f056995e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05640c21d0 con 0x7f056415e4d0 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.343+0000 7f056995e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f05640c3c30 con 0x7f05640c69b0 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.344+0000 7f0562ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05640c69b0 0x7f0564163710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.346+0000 7f0562ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05640c69b0 0x7f0564163710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:33042/0 (socket says 172.21.15.88:33042) 2024-09-21T02:03:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.346+0000 7f0562ffd640 1 -- 172.21.15.88:0/1105565925 learned_addr learned my addr 172.21.15.88:0/1105565925 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.348+0000 7f2801694640 1 --2- 172.21.15.88:0/1054366902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27fc0cd000 0x7f27fc153bf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27ec02aa50 tx=0x7f27ec030570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.348+0000 7f0562ffd640 1 -- 172.21.15.88:0/1105565925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f056415e4d0 msgr2=0x7f056415e8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.348+0000 7f0562ffd640 1 --2- 172.21.15.88:0/1105565925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f056415e4d0 0x7f056415e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.348+0000 7f0562ffd640 1 -- 172.21.15.88:0/1105565925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05640c5cb0 msgr2=0x7f05640cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.348+0000 7f0562ffd640 1 --2- 172.21.15.88:0/1105565925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05640c5cb0 0x7f05640cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.348+0000 7f0562ffd640 1 -- 172.21.15.88:0/1105565925 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05640c4840 con 0x7f05640c69b0 2024-09-21T02:03:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.351+0000 7f0562ffd640 1 --2- 172.21.15.88:0/1105565925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05640c69b0 0x7f0564163710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f055c02ad90 tx=0x7f055c006d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.351+0000 7f054bfff640 1 -- 172.21.15.88:0/1105565925 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c044070 con 0x7f05640c69b0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.351+0000 7f054bfff640 1 -- 172.21.15.88:0/1105565925 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f055c006420 con 0x7f05640c69b0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.351+0000 7f054bfff640 1 -- 172.21.15.88:0/1105565925 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c006870 con 0x7f05640c69b0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.352+0000 7f056995e640 1 -- 172.21.15.88:0/1105565925 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05640c4b20 con 0x7f05640c69b0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.352+0000 7f056995e640 1 -- 172.21.15.88:0/1105565925 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05640c54b0 con 0x7f05640c69b0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.355+0000 7f054bfff640 1 -- 172.21.15.88:0/1105565925 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f055c047070 con 0x7f05640c69b0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.355+0000 7f054bfff640 1 -- 172.21.15.88:0/1105565925 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f055c049810 con 0x7f05640c69b0 2024-09-21T02:03:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.355+0000 7f0549ffb640 1 -- 172.21.15.88:0/1105565925 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0544005290 con 0x7f05640c69b0 2024-09-21T02:03:33.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.357+0000 7f31a9ffb640 1 -- 172.21.15.88:0/914603780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f31a4005370 con 0x7f31c41f62f0 2024-09-21T02:03:33.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f2145bed640 1 -- 172.21.15.88:0/413077186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f214015bb90 msgr2=0x7f214015bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f2145bed640 1 --2- 172.21.15.88:0/413077186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f214015bb90 0x7f214015bfa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f21400b2e00 tx=0x7f213802a810 comp rx=0 tx=0).stop 2024-09-21T02:03:33.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f320359e640 1 -- 172.21.15.88:0/2311165417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f31e8005370 con 0x7f31fc0df620 2024-09-21T02:03:33.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f2145bed640 1 -- 172.21.15.88:0/413077186 shutdown_connections 2024-09-21T02:03:33.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f2145bed640 1 --2- 172.21.15.88:0/413077186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f214015bb90 0x7f214015bfa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f2145bed640 1 --2- 172.21.15.88:0/413077186 >> v2:172.21.15.88:3300/0 conn(0x7f21400c2220 0x7f21400c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f2145bed640 1 --2- 172.21.15.88:0/413077186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21400af860 0x7f21401556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.358+0000 7f2145bed640 1 -- 172.21.15.88:0/413077186 >> 172.21.15.88:0/413077186 conn(0x7f214006cd60 msgr2=0x7f214006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 -- 172.21.15.88:0/2094567131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164072bf0 msgr2=0x7f0164073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 --2- 172.21.15.88:0/2094567131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164072bf0 0x7f0164073600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01640734b0 tx=0x7f015402a540 comp rx=0 tx=0).stop 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 -- 172.21.15.88:0/2094567131 shutdown_connections 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 --2- 172.21.15.88:0/2094567131 >> v2:172.21.15.88:3300/0 conn(0x7f0164073b40 0x7f01640c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 --2- 172.21.15.88:0/2094567131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164072bf0 0x7f0164073600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 -- 172.21.15.88:0/2094567131 >> 172.21.15.88:0/2094567131 conn(0x7f016406cf70 msgr2=0x7f016406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 -- 172.21.15.88:0/2094567131 shutdown_connections 2024-09-21T02:03:33.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7fa155ffb640 1 -- 172.21.15.88:0/1601429365 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa15001f020 con 0x7fa1601536a0 2024-09-21T02:03:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.359+0000 7f054bfff640 1 -- 172.21.15.88:0/1105565925 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f055c04e020 con 0x7f05640c69b0 2024-09-21T02:03:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.364+0000 7f2145bed640 1 -- 172.21.15.88:0/413077186 shutdown_connections 2024-09-21T02:03:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.364+0000 7f2145bed640 1 -- 172.21.15.88:0/413077186 wait complete. 2024-09-21T02:03:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.365+0000 7f22eaffd640 1 -- 172.21.15.88:0/1690461570 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f22e401c460 con 0x7f22f4152840 2024-09-21T02:03:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.365+0000 7f22eaffd640 1 -- 172.21.15.88:0/1690461570 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22e40266e0 con 0x7f22f4152840 2024-09-21T02:03:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.365+0000 7f22eaffd640 1 -- 172.21.15.88:0/1690461570 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f22e4026ae0 con 0x7f22f4152840 2024-09-21T02:03:33.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.362+0000 7f016b073640 1 -- 172.21.15.88:0/2094567131 wait complete. 2024-09-21T02:03:33.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.365+0000 7f016b073640 1 Processor -- start 2024-09-21T02:03:33.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.365+0000 7f016b073640 1 -- start start 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.367+0000 7f016b073640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0164072bf0 0x7f0164159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.368+0000 7f016b073640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164073b40 0x7f016415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.368+0000 7f016b073640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01640c6990 0x7f0164157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.368+0000 7f016b073640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01640c21d0 con 0x7f01640c6990 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.368+0000 7f016b073640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01640c1ef0 con 0x7f0164073b40 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.368+0000 7f016b073640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f01640c24b0 con 0x7f0164072bf0 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.369+0000 7f016a071640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0164072bf0 0x7f0164159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.369+0000 7f016a071640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0164072bf0 0x7f0164159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:33092/0 (socket says 172.21.15.88:33092) 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.369+0000 7f016a071640 1 -- 172.21.15.88:0/2935976789 learned_addr learned my addr 172.21.15.88:0/2935976789 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.369+0000 7f0169870640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164073b40 0x7f016415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f27f27fc640 1 -- 172.21.15.88:0/1054366902 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec040030 con 0x7f27fc0cd000 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f2802e97640 1 -- 172.21.15.88:0/1054366902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc1f6c30 con 0x7f27fc0cd000 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f2802e97640 1 -- 172.21.15.88:0/1054366902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc1f71f0 con 0x7f27fc0cd000 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.368+0000 7f27f27fc640 1 -- 172.21.15.88:0/1054366902 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27ec030b20 con 0x7f27fc0cd000 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.369+0000 7f27f27fc640 1 -- 172.21.15.88:0/1054366902 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec037040 con 0x7f27fc0cd000 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f27e7fff640 1 -- 172.21.15.88:0/1054366902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc005290 con 0x7f27fc0cd000 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f53d8a1c640 1 -- 172.21.15.88:0/207821098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd440 msgr2=0x7f53d416c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f53d8a1c640 1 --2- 172.21.15.88:0/207821098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd440 0x7f53d416c6a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f53d40c7ad0 tx=0x7f53c802b1c0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f53d8a1c640 1 -- 172.21.15.88:0/207821098 shutdown_connections 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f53d8a1c640 1 --2- 172.21.15.88:0/207821098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd440 0x7f53d416c6a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f53d8a1c640 1 --2- 172.21.15.88:0/207821098 >> v2:172.21.15.88:3300/0 conn(0x7f53d40cd030 0x7f53d40737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.366+0000 7f53d8a1c640 1 -- 172.21.15.88:0/207821098 >> 172.21.15.88:0/207821098 conn(0x7f53d406cf70 msgr2=0x7f53d406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.367+0000 7f2145bed640 1 Processor -- start 2024-09-21T02:03:33.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.367+0000 7f2145bed640 1 -- start start 2024-09-21T02:03:33.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f2145bed640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21400af860 0x7f21400b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f016a071640 1 -- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164073b40 msgr2=0x7f016415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f016a071640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164073b40 0x7f016415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f016a071640 1 -- 172.21.15.88:0/2935976789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01640c6990 msgr2=0x7f0164157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f016a071640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01640c6990 0x7f0164157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f016a071640 1 -- 172.21.15.88:0/2935976789 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016415b370 con 0x7f0164072bf0 2024-09-21T02:03:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f016a071640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0164072bf0 0x7f0164159f70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f016000c370 tx=0x7f016000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f85ce4ce640 1 -- 172.21.15.88:0/254601245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00469d0 msgr2=0x7f85c00db6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f85ce4ce640 1 --2- 172.21.15.88:0/254601245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00469d0 0x7f85c00db6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f85c00466c0 tx=0x7f85b402b250 comp rx=0 tx=0).stop 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f85ce4ce640 1 -- 172.21.15.88:0/254601245 shutdown_connections 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f85ce4ce640 1 --2- 172.21.15.88:0/254601245 >> v2:172.21.15.195:3300/0 conn(0x7f85c00d16a0 0x7f85c0005c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f85ce4ce640 1 --2- 172.21.15.88:0/254601245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00469d0 0x7f85c00db6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f85ce4ce640 1 -- 172.21.15.88:0/254601245 >> 172.21.15.88:0/254601245 conn(0x7f85c004fe30 msgr2=0x7f85c0050240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.379+0000 7f85ce4ce640 1 -- 172.21.15.88:0/254601245 shutdown_connections 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f310a7fc640 1 -- 172.21.15.88:0/3174784991 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104004590 con 0x7f310c15c370 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f310a7fc640 1 -- 172.21.15.88:0/3174784991 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3104004820 con 0x7f310c15c370 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c153100 con 0x7f310c15c370 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c154750 con 0x7f310c15c370 2024-09-21T02:03:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.371+0000 7f310a7fc640 1 -- 172.21.15.88:0/3174784991 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310403a680 con 0x7f310c15c370 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.378+0000 7f310a7fc640 1 -- 172.21.15.88:0/3174784991 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3104042020 con 0x7f310c15c370 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f53d8a1c640 1 -- 172.21.15.88:0/207821098 shutdown_connections 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f53d8a1c640 1 -- 172.21.15.88:0/207821098 wait complete. 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.373+0000 7f53d8a1c640 1 Processor -- start 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.373+0000 7f53d8a1c640 1 -- start start 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f53d8a1c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53d4073d10 0x7f53d4159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f53d8a1c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd030 0x7f53d4167f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f53d37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53d4073d10 0x7f53d4159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f53d37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53d4073d10 0x7f53d4159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:33116/0 (socket says 172.21.15.88:33116) 2024-09-21T02:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f53d2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd030 0x7f53d4167f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd030 0x7f53d4167f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:47378/0 (socket says 172.21.15.88:47378) 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d8a1c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53d40c4300 0x7f53d40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d37fe640 1 -- 172.21.15.88:0/2008303010 learned_addr learned my addr 172.21.15.88:0/2008303010 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d8a1c640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4072bf0 con 0x7f53d40c4300 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d8a1c640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f53d40c8300 con 0x7f53d40cd030 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d8a1c640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f53d4072d70 con 0x7f53d4073d10 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d2ffd640 1 -- 172.21.15.88:0/2008303010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53d4073d10 msgr2=0x7f53d4159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d2ffd640 1 --2- 172.21.15.88:0/2008303010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53d4073d10 0x7f53d4159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d2ffd640 1 -- 172.21.15.88:0/2008303010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53d40c4300 msgr2=0x7f53d40c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d2ffd640 1 --2- 172.21.15.88:0/2008303010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53d40c4300 0x7f53d40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d2ffd640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4156a70 con 0x7f53d40cd030 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f53d2ffd640 1 --2- 172.21.15.88:0/2008303010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd030 0x7f53d4167f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00ca50 tx=0x7f53cc00cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.377+0000 7f53bbfff640 1 -- 172.21.15.88:0/2008303010 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc016020 con 0x7f53d40cd030 2024-09-21T02:03:33.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.377+0000 7f53bbfff640 1 -- 172.21.15.88:0/2008303010 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53cc00aca0 con 0x7f53d40cd030 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.377+0000 7f53d8a1c640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d415c4b0 con 0x7f53d40cd030 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.377+0000 7f53d8a1c640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d40c6d60 con 0x7f53d40cd030 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.378+0000 7f53bbfff640 1 -- 172.21.15.88:0/2008303010 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc012680 con 0x7f53d40cd030 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.380+0000 7f53bbfff640 1 -- 172.21.15.88:0/2008303010 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53cc019020 con 0x7f53d40cd030 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.380+0000 7f53bbfff640 1 -- 172.21.15.88:0/2008303010 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53cc01bc60 con 0x7f53d40cd030 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f2145bed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21400c2220 0x7f21400c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f2145bed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f214015bb90 0x7f21400c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f2145bed640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21400c83a0 con 0x7f21400af860 2024-09-21T02:03:33.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f2145bed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f21400c80c0 con 0x7f21400c2220 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.370+0000 7f2145bed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f214015c4e0 con 0x7f214015bb90 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f214015bb90 0x7f21400c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f214015bb90 0x7f21400c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:33104/0 (socket says 172.21.15.88:33104) 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 -- 172.21.15.88:0/1423554986 learned_addr learned my addr 172.21.15.88:0/1423554986 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213effd640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21400c2220 0x7f21400c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 -- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21400c2220 msgr2=0x7f21400c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21400c2220 0x7f21400c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 -- 172.21.15.88:0/1423554986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21400af860 msgr2=0x7f21400b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21400af860 0x7f21400b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.372+0000 7f213ffff640 1 -- 172.21.15.88:0/1423554986 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21400b06e0 con 0x7f214015bb90 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.374+0000 7f213ffff640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f214015bb90 0x7f21400c9cf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f213800b160 tx=0x7f2138008d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.375+0000 7f212bfff640 1 -- 172.21.15.88:0/1423554986 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138042020 con 0x7f214015bb90 2024-09-21T02:03:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f212bfff640 1 -- 172.21.15.88:0/1423554986 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2138038040 con 0x7f214015bb90 2024-09-21T02:03:33.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f2145bed640 1 -- 172.21.15.88:0/1423554986 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21400ca820 con 0x7f214015bb90 2024-09-21T02:03:33.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.376+0000 7f2145bed640 1 -- 172.21.15.88:0/1423554986 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21400b08f0 con 0x7f214015bb90 2024-09-21T02:03:33.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.377+0000 7f212bfff640 1 -- 172.21.15.88:0/1423554986 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138037050 con 0x7f214015bb90 2024-09-21T02:03:33.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.380+0000 7f53b9ffb640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b4005290 con 0x7f53d40cd030 2024-09-21T02:03:33.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.379+0000 7f85ce4ce640 1 -- 172.21.15.88:0/254601245 wait complete. 2024-09-21T02:03:33.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.384+0000 7f212bfff640 1 -- 172.21.15.88:0/1423554986 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2138045070 con 0x7f214015bb90 2024-09-21T02:03:33.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.381+0000 7f85ce4ce640 1 Processor -- start 2024-09-21T02:03:33.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.384+0000 7f85ce4ce640 1 -- start start 2024-09-21T02:03:33.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.385+0000 7f015affd640 1 -- 172.21.15.88:0/2935976789 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160006890 con 0x7f0164072bf0 2024-09-21T02:03:33.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.385+0000 7f016b073640 1 -- 172.21.15.88:0/2935976789 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0164202920 con 0x7f0164072bf0 2024-09-21T02:03:33.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.385+0000 7f31abfff640 1 -- 172.21.15.88:0/914603780 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f31b4035430 con 0x7f31c41f62f0 2024-09-21T02:03:33.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.385+0000 7f016b073640 1 -- 172.21.15.88:0/2935976789 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0164159770 con 0x7f0164072bf0 2024-09-21T02:03:33.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.385+0000 7f27f27fc640 1 -- 172.21.15.88:0/1054366902 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f27ec03a070 con 0x7f27fc0cd000 2024-09-21T02:03:33.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.385+0000 7f27f27fc640 1 -- 172.21.15.88:0/1054366902 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f27ec005020 con 0x7f27fc0cd000 2024-09-21T02:03:33.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f27f27fc640 1 -- 172.21.15.88:0/1054366902 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f27ec004020 con 0x7f27fc0cd000 2024-09-21T02:03:33.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f31ca132640 1 -- 172.21.15.88:0/914603780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c41f62f0 msgr2=0x7f31c4150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f31ca132640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c41f62f0 0x7f31c4150260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31b4004060 tx=0x7f31b4003c80 comp rx=0 tx=0).stop 2024-09-21T02:03:33.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f310a7fc640 1 -- 172.21.15.88:0/3174784991 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3104044410 con 0x7f310c15c370 2024-09-21T02:03:33.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec005290 con 0x7f310c15c370 2024-09-21T02:03:33.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.388+0000 7f015affd640 1 -- 172.21.15.88:0/2935976789 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0160006b20 con 0x7f0164072bf0 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.388+0000 7f015affd640 1 -- 172.21.15.88:0/2935976789 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160007020 con 0x7f0164072bf0 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.389+0000 7f015affd640 1 -- 172.21.15.88:0/2935976789 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0160019020 con 0x7f0164072bf0 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.389+0000 7f015affd640 1 -- 172.21.15.88:0/2935976789 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0160012460 con 0x7f0164072bf0 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f31ca132640 1 -- 172.21.15.88:0/914603780 shutdown_connections 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f31ca132640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c4150eb0 0x7f31c415f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f31ca132640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c4155590 0x7f31c415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f31ca132640 1 --2- 172.21.15.88:0/914603780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c41f62f0 0x7f31c4150260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.386+0000 7f31ca132640 1 -- 172.21.15.88:0/914603780 >> 172.21.15.88:0/914603780 conn(0x7f31c406d390 msgr2=0x7f31c4076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.387+0000 7f31ca132640 1 -- 172.21.15.88:0/914603780 shutdown_connections 2024-09-21T02:03:33.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.387+0000 7f31ca132640 1 -- 172.21.15.88:0/914603780 wait complete. 2024-09-21T02:03:33.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.388+0000 7f212bfff640 1 -- 172.21.15.88:0/1423554986 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f21380055f0 con 0x7f214015bb90 2024-09-21T02:03:33.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.387+0000 7f63ef3c6640 1 -- 172.21.15.88:0/3434583184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d4004d40 msgr2=0x7f63d4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.387+0000 7f63ef3c6640 1 --2- 172.21.15.88:0/3434583184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63d4004d40 0x7f63d4005150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63e000dc80 tx=0x7f63d4005830 comp rx=0 tx=0).stop 2024-09-21T02:03:33.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f0158ff9640 1 -- 172.21.15.88:0/2935976789 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c005290 con 0x7f0164072bf0 2024-09-21T02:03:33.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.388+0000 7f2129ffb640 1 -- 172.21.15.88:0/1423554986 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2120005290 con 0x7f214015bb90 2024-09-21T02:03:33.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f63ed963640 1 -- 172.21.15.88:0/3434583184 reap_dead start 2024-09-21T02:03:33.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f63ef3c6640 1 -- 172.21.15.88:0/3434583184 shutdown_connections 2024-09-21T02:03:33.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f63ef3c6640 1 -- 172.21.15.88:0/3434583184 >> 172.21.15.88:0/3434583184 conn(0x7f63e004fe60 msgr2=0x7f63e0050270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85ce4ce640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c0044cb0 0x7f85c00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85ce4ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c00469d0 0x7f85c01776e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85ce4ce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00d16a0 0x7f85c0005b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c00469d0 0x7f85c01776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c00469d0 0x7f85c01776e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:47390/0 (socket says 172.21.15.88:47390) 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85c77fe640 1 -- 172.21.15.88:0/388709758 learned_addr learned my addr 172.21.15.88:0/388709758 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85ce4ce640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0045ef0 con 0x7f85c0044cb0 2024-09-21T02:03:33.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.394+0000 7f63ef3c6640 1 -- 172.21.15.88:0/3434583184 shutdown_connections 2024-09-21T02:03:33.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.390+0000 7f85cca6b640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00d16a0 0x7f85c0005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.394+0000 7f85ce4ce640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85c0045c10 con 0x7f85c00469d0 2024-09-21T02:03:33.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.394+0000 7f85ce4ce640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f85c00461d0 con 0x7f85c00d16a0 2024-09-21T02:03:33.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.394+0000 7f212bfff640 1 -- 172.21.15.88:0/1423554986 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2138008e90 con 0x7f214015bb90 2024-09-21T02:03:33.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.395+0000 7f63ef3c6640 1 -- 172.21.15.88:0/3434583184 wait complete. 2024-09-21T02:03:33.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.399+0000 7f015affd640 1 -- 172.21.15.88:0/2935976789 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0160011590 con 0x7f0164072bf0 2024-09-21T02:03:33.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.395+0000 7f85c77fe640 1 -- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00d16a0 msgr2=0x7f85c0005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.399+0000 7f31f2ffd640 1 -- 172.21.15.88:0/2311165417 <== mon.1 v2:172.21.15.195:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v10)=0 v10) ==== 127+0+0 (secure 0 0 0) 0x7f31f80215f0 con 0x7f31fc0df620 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.395+0000 7f85c77fe640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00d16a0 0x7f85c0005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.395+0000 7f85c77fe640 1 -- 172.21.15.88:0/388709758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c0044cb0 msgr2=0x7f85c00cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.395+0000 7f85c77fe640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c0044cb0 0x7f85c00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.395+0000 7f85c77fe640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c00cdbc0 con 0x7f85c00469d0 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.399+0000 7f85c77fe640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c00469d0 0x7f85c01776e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85b4007440 tx=0x7f85b4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.399+0000 7f85c4ff9640 1 -- 172.21.15.88:0/388709758 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4003c20 con 0x7f85c00469d0 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.400+0000 7f85c4ff9640 1 -- 172.21.15.88:0/388709758 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85b4005020 con 0x7f85c00469d0 2024-09-21T02:03:33.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.400+0000 7f85ce4ce640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c0177f20 con 0x7f85c00469d0 2024-09-21T02:03:33.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.400+0000 7f53bbfff640 1 -- 172.21.15.88:0/2008303010 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f53cc020020 con 0x7f53d40cd030 2024-09-21T02:03:33.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.400+0000 7f85ce4ce640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c00cef80 con 0x7f85c00469d0 2024-09-21T02:03:33.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.401+0000 7f85b27fc640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8005290 con 0x7f85c00469d0 2024-09-21T02:03:33.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.402+0000 7f85c4ff9640 1 -- 172.21.15.88:0/388709758 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4005470 con 0x7f85c00469d0 2024-09-21T02:03:33.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.402+0000 7f85c4ff9640 1 -- 172.21.15.88:0/388709758 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85b4044030 con 0x7f85c00469d0 2024-09-21T02:03:33.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.402+0000 7f85c4ff9640 1 -- 172.21.15.88:0/388709758 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f85b403d420 con 0x7f85c00469d0 2024-09-21T02:03:33.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.398+0000 7f63ef3c6640 1 Processor -- start 2024-09-21T02:03:33.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f85c4ff9640 1 -- 172.21.15.88:0/388709758 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f85b404b020 con 0x7f85c00469d0 2024-09-21T02:03:33.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.403+0000 7f63ef3c6640 1 -- start start 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ef3c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63e00cea90 0x7f63e017cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ef3c6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e017d4e0 0x7f63e00cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ef3c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e00d0200 0x7f63e0048d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ef3c6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00468e0 con 0x7f63e017d4e0 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ef3c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00d9320 con 0x7f63e00d0200 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ef3c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f63e00458f0 con 0x7f63e00cea90 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ec961640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e017d4e0 0x7f63e00cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ec961640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e017d4e0 0x7f63e00cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41324/0 (socket says 172.21.15.88:41324) 2024-09-21T02:03:33.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.405+0000 7f63ec961640 1 -- 172.21.15.88:0/182852062 learned_addr learned my addr 172.21.15.88:0/182852062 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f31f0ff9640 1 -- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0df620 msgr2=0x7f31fc049860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f31f0ff9640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0df620 0x7f31fc049860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f31f800e7c0 tx=0x7f31f800ec80 comp rx=0 tx=0).stop 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f31f0ff9640 1 -- 172.21.15.88:0/2311165417 shutdown_connections 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f31f0ff9640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31fc0d0a70 0x7f31fc04ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f31f0ff9640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0df620 0x7f31fc049860 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f31f0ff9640 1 --2- 172.21.15.88:0/2311165417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc044d20 0x7f31fc0d4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f31f0ff9640 1 -- 172.21.15.88:0/2311165417 >> 172.21.15.88:0/2311165417 conn(0x7f31fc04fe30 msgr2=0x7f31fc0dc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f63ec961640 1 -- 172.21.15.88:0/182852062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63e00cea90 msgr2=0x7f63e017cfa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f63ec961640 1 --2- 172.21.15.88:0/182852062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63e00cea90 0x7f63e017cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f63ec961640 1 -- 172.21.15.88:0/182852062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e00d0200 msgr2=0x7f63e0048d30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:03:33.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f63ec961640 1 --2- 172.21.15.88:0/182852062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e00d0200 0x7f63e0048d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.406+0000 7f63ec961640 1 -- 172.21.15.88:0/182852062 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e017d8f0 con 0x7f63e017d4e0 2024-09-21T02:03:33.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.410+0000 7f31f0ff9640 1 -- 172.21.15.88:0/2311165417 shutdown_connections 2024-09-21T02:03:33.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.410+0000 7f31f0ff9640 1 -- 172.21.15.88:0/2311165417 wait complete. 2024-09-21T02:03:33.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.410+0000 7f64df7fe640 1 -- 172.21.15.88:0/2777986799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f64d4005370 con 0x7f64f40b2f80 2024-09-21T02:03:33.414 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.414+0000 7f310a7fc640 1 -- 172.21.15.88:0/3174784991 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3104052020 con 0x7f310c15c370 2024-09-21T02:03:33.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.414+0000 7f63ec961640 1 --2- 172.21.15.88:0/182852062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e017d4e0 0x7f63e00cfcc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63d800b5b0 tx=0x7f63d800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.418+0000 7f63ddffb640 1 -- 172.21.15.88:0/182852062 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8014020 con 0x7f63e017d4e0 2024-09-21T02:03:33.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.418+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e00475c0 con 0x7f63e017d4e0 2024-09-21T02:03:33.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.418+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e00cf4c0 con 0x7f63e017d4e0 2024-09-21T02:03:33.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.419+0000 7f63ddffb640 1 -- 172.21.15.88:0/182852062 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63d8003700 con 0x7f63e017d4e0 2024-09-21T02:03:33.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.419+0000 7f63ddffb640 1 -- 172.21.15.88:0/182852062 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8003b50 con 0x7f63e017d4e0 2024-09-21T02:03:33.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.423+0000 7f64e9ffb640 1 -- 172.21.15.88:0/2777986799 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v11)=0 v11) ==== 116+0+0 (secure 0 0 0) 0x7f64e4014440 con 0x7f64f40b2f80 2024-09-21T02:03:33.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f40b2f80 msgr2=0x7f64f41fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f40b2f80 0x7f64f41fef20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b7b0 tx=0x7f64e400bc70 comp rx=0 tx=0).stop 2024-09-21T02:03:33.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f63ddffb640 1 -- 172.21.15.88:0/182852062 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f63d8017020 con 0x7f63e017d4e0 2024-09-21T02:03:33.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f63ddffb640 1 -- 172.21.15.88:0/182852062 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63d80107c0 con 0x7f63e017d4e0 2024-09-21T02:03:33.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e004b580 con 0x7f63e017d4e0 2024-09-21T02:03:33.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 shutdown_connections 2024-09-21T02:03:33.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f415c370 0x7f64f40b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f40b2f80 0x7f64f41fef20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 --2- 172.21.15.88:0/2777986799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f4070540 0x7f64f42025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 >> 172.21.15.88:0/2777986799 conn(0x7f64f406d390 msgr2=0x7f64f40c7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.425+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 shutdown_connections 2024-09-21T02:03:33.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.426+0000 7f64fb7b0640 1 -- 172.21.15.88:0/2777986799 wait complete. 2024-09-21T02:03:33.433 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.436+0000 7f63ddffb640 1 -- 172.21.15.88:0/182852062 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f63d801e020 con 0x7f63e017d4e0 2024-09-21T02:03:33.454 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.538+0000 7f1369ffb640 1 -- 172.21.15.88:0/1078432223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f1360005580 con 0x7f13801521a0 2024-09-21T02:03:33.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.547+0000 7f136bfff640 1 -- 172.21.15.88:0/1078432223 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v12)=0 v12) ==== 115+0+0 (secure 0 0 0) 0x7f136c043d20 con 0x7f13801521a0 2024-09-21T02:03:33.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.557+0000 7f1369ffb640 1 -- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13801521a0 msgr2=0x7f13801570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.557+0000 7f1369ffb640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13801521a0 0x7f13801570a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f138006f940 tx=0x7f136c0038a0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.562+0000 7f1369ffb640 1 -- 172.21.15.88:0/1078432223 shutdown_connections 2024-09-21T02:03:33.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.562+0000 7f1369ffb640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f138015b720 0x7f138013db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.562+0000 7f1369ffb640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f13801521a0 0x7f13801570a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.562+0000 7f1369ffb640 1 --2- 172.21.15.88:0/1078432223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13800c2550 0x7f1380141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.562+0000 7f1369ffb640 1 -- 172.21.15.88:0/1078432223 >> 172.21.15.88:0/1078432223 conn(0x7f138006cd60 msgr2=0x7f1380158290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.562+0000 7f1369ffb640 1 -- 172.21.15.88:0/1078432223 shutdown_connections 2024-09-21T02:03:33.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.562+0000 7f1369ffb640 1 -- 172.21.15.88:0/1078432223 wait complete. 2024-09-21T02:03:33.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.586+0000 7fca78ff9640 1 -- 172.21.15.88:0/371328589 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fca64005580 con 0x7fca8c070950 2024-09-21T02:03:33.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.575+0000 7f806e7fc640 1 -- 172.21.15.88:0/3455467713 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f8064005580 con 0x7f807c0fd0e0 2024-09-21T02:03:33.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.592+0000 7fca7affd640 1 -- 172.21.15.88:0/371328589 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v13)=0 v13) ==== 120+0+0 (secure 0 0 0) 0x7fca7c010740 con 0x7fca8c070950 2024-09-21T02:03:33.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 -- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070950 msgr2=0x7fca8c0cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070950 0x7fca8c0cbe50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00bfd0 tx=0x7fca7c00da70 comp rx=0 tx=0).stop 2024-09-21T02:03:33.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 -- 172.21.15.88:0/371328589 shutdown_connections 2024-09-21T02:03:33.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0c6d00 0x7fca8c149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca8c0b2f80 0x7fca8c14e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 --2- 172.21.15.88:0/371328589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca8c070950 0x7fca8c0cbe50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 -- 172.21.15.88:0/371328589 >> 172.21.15.88:0/371328589 conn(0x7fca8c06d390 msgr2=0x7fca8c14fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 -- 172.21.15.88:0/371328589 shutdown_connections 2024-09-21T02:03:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.595+0000 7fca90d20640 1 -- 172.21.15.88:0/371328589 wait complete. 2024-09-21T02:03:33.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.601+0000 7f0549ffb640 1 -- 172.21.15.88:0/1105565925 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f0544005580 con 0x7f05640c69b0 2024-09-21T02:03:33.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.611+0000 7f3658ff9640 1 -- 172.21.15.88:0/156714292 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_dir_prefetch}] v 0) -- 0x7f366c15a120 con 0x7f366c070540 2024-09-21T02:03:33.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.612+0000 7fa14b7fe640 1 -- 172.21.15.88:0/1601429365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fa140005580 con 0x7fa1601536a0 2024-09-21T02:03:33.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.614+0000 7f8080ff9640 1 -- 172.21.15.88:0/3455467713 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7f80780103c0 con 0x7f807c0fd0e0 2024-09-21T02:03:33.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.615+0000 7f6d917fa640 1 -- 172.21.15.88:0/2709451139 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f6d8c005580 con 0x7f6dac0c7560 2024-09-21T02:03:33.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.616+0000 7f806e7fc640 1 -- 172.21.15.88:0/3455467713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f807c0fd0e0 msgr2=0x7f807c0fd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.616+0000 7f806e7fc640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f807c0fd0e0 0x7f807c0fd970 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8078006680 tx=0x7f80780066b0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.621+0000 7f806e7fc640 1 -- 172.21.15.88:0/3455467713 shutdown_connections 2024-09-21T02:03:33.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.621+0000 7f806e7fc640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f807c0f8100 0x7f807c0fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.621+0000 7f806e7fc640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f807c0f7cf0 0x7f807c0fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.621+0000 7f806e7fc640 1 --2- 172.21.15.88:0/3455467713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f807c0fd0e0 0x7f807c0fd970 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.621+0000 7f806e7fc640 1 -- 172.21.15.88:0/3455467713 >> 172.21.15.88:0/3455467713 conn(0x7f807c04fe30 msgr2=0x7f807c034510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.622+0000 7f806e7fc640 1 -- 172.21.15.88:0/3455467713 shutdown_connections 2024-09-21T02:03:33.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.622+0000 7f806e7fc640 1 -- 172.21.15.88:0/3455467713 wait complete. 2024-09-21T02:03:33.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.624+0000 7f365affd640 1 -- 172.21.15.88:0/156714292 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_dir_prefetch}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7f365c028440 con 0x7f366c070540 2024-09-21T02:03:33.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.625+0000 7f3658ff9640 1 -- 172.21.15.88:0/156714292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f366c070540 msgr2=0x7f366c159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.625+0000 7f3658ff9640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f366c070540 0x7f366c159b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f365c007890 tx=0x7f365c007d50 comp rx=0 tx=0).stop 2024-09-21T02:03:33.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.626+0000 7f3658ff9640 1 -- 172.21.15.88:0/156714292 shutdown_connections 2024-09-21T02:03:33.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.626+0000 7f3658ff9640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f366c0c7560 0x7f366c0c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.626+0000 7f3658ff9640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f366c070950 0x7f366c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.626+0000 7f3658ff9640 1 --2- 172.21.15.88:0/156714292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f366c070540 0x7f366c159b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.626+0000 7f3658ff9640 1 -- 172.21.15.88:0/156714292 >> 172.21.15.88:0/156714292 conn(0x7f366c06d390 msgr2=0x7f366c0c7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.626+0000 7f3658ff9640 1 -- 172.21.15.88:0/156714292 shutdown_connections 2024-09-21T02:03:33.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.626+0000 7f3658ff9640 1 -- 172.21.15.88:0/156714292 wait complete. 2024-09-21T02:03:33.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.632+0000 7fd4b4ff9640 1 -- 172.21.15.88:0/229097348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fd4ac005580 con 0x7fd4c80b2f80 2024-09-21T02:03:33.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.638+0000 7fa155ffb640 1 -- 172.21.15.88:0/1601429365 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7fa150009170 con 0x7fa1601536a0 2024-09-21T02:03:33.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.639+0000 7fa1672b4640 1 -- 172.21.15.88:0/1601429365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1601536a0 msgr2=0x7fa160153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.639+0000 7fa1672b4640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1601536a0 0x7fa160153ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b390 tx=0x7fa15000b850 comp rx=0 tx=0).stop 2024-09-21T02:03:33.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.639+0000 7fa1672b4640 1 -- 172.21.15.88:0/1601429365 shutdown_connections 2024-09-21T02:03:33.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.639+0000 7fa1672b4640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1601f9b70 0x7fa1601f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.639+0000 7fa1672b4640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1601536a0 0x7fa160153ab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.639+0000 7fa1672b4640 1 --2- 172.21.15.88:0/1601429365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa16015d1a0 0x7fa160158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.639+0000 7fa1672b4640 1 -- 172.21.15.88:0/1601429365 >> 172.21.15.88:0/1601429365 conn(0x7fa16006d390 msgr2=0x7fa1600cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.640+0000 7fa1672b4640 1 -- 172.21.15.88:0/1601429365 shutdown_connections 2024-09-21T02:03:33.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.646+0000 7fa1672b4640 1 -- 172.21.15.88:0/1601429365 wait complete. 2024-09-21T02:03:33.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.649+0000 7f054bfff640 1 -- 172.21.15.88:0/1105565925 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v17)=0 v17) ==== 119+0+0 (secure 0 0 0) 0x7f055c049a60 con 0x7f05640c69b0 2024-09-21T02:03:33.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.651+0000 7f056995e640 1 -- 172.21.15.88:0/1105565925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05640c69b0 msgr2=0x7f0564163710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.651+0000 7f056995e640 1 --2- 172.21.15.88:0/1105565925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05640c69b0 0x7f0564163710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f055c02ad90 tx=0x7f055c006d40 comp rx=0 tx=0).stop 2024-09-21T02:03:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.651+0000 7f056995e640 1 -- 172.21.15.88:0/1105565925 shutdown_connections 2024-09-21T02:03:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.651+0000 7f056995e640 1 --2- 172.21.15.88:0/1105565925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f056415e4d0 0x7f056415e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.651+0000 7f056995e640 1 --2- 172.21.15.88:0/1105565925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05640c5cb0 0x7f05640cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.651+0000 7f056995e640 1 --2- 172.21.15.88:0/1105565925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05640c69b0 0x7f0564163710 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.651+0000 7f056995e640 1 -- 172.21.15.88:0/1105565925 >> 172.21.15.88:0/1105565925 conn(0x7f056406d390 msgr2=0x7f056416c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.652+0000 7f056995e640 1 -- 172.21.15.88:0/1105565925 shutdown_connections 2024-09-21T02:03:33.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.652+0000 7f056995e640 1 -- 172.21.15.88:0/1105565925 wait complete. 2024-09-21T02:03:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.670+0000 7f6d937fe640 1 -- 172.21.15.88:0/2709451139 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7f6d9c031940 con 0x7f6dac0c7560 2024-09-21T02:03:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac0c7560 msgr2=0x7f6dac0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac0c7560 0x7f6dac0cc7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009920 tx=0x7f6d9c003820 comp rx=0 tx=0).stop 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 shutdown_connections 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dac16a1b0 0x7f6dac15b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dac0c7560 0x7f6dac0cc7e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 --2- 172.21.15.88:0/2709451139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dac070950 0x7f6dac1692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 >> 172.21.15.88:0/2709451139 conn(0x7f6dac06d390 msgr2=0x7f6dac161080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.671+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 shutdown_connections 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.672+0000 7f6db11eb640 1 -- 172.21.15.88:0/2709451139 wait complete. 2024-09-21T02:03:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.672+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f1250005580 con 0x7f12700c6810 2024-09-21T02:03:33.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.680+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f30ec005580 con 0x7f310c15c370 2024-09-21T02:03:33.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.679+0000 7f53b9ffb640 1 -- 172.21.15.88:0/2008303010 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f53b4005580 con 0x7f53d40cd030 2024-09-21T02:03:33.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.680+0000 7f2129ffb640 1 -- 172.21.15.88:0/1423554986 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f2120005580 con 0x7f214015bb90 2024-09-21T02:03:33.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.685+0000 7f0158ff9640 1 -- 172.21.15.88:0/2935976789 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f014c005580 con 0x7f0164072bf0 2024-09-21T02:03:33.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.691+0000 7f22e8ff9640 1 -- 172.21.15.88:0/1690461570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f22f40c7ab0 con 0x7f22f4152840 2024-09-21T02:03:33.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.692+0000 7fd4b6ffd640 1 -- 172.21.15.88:0/229097348 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v19)=0 v19) ==== 118+0+0 (secure 0 0 0) 0x7fd4c001ecc0 con 0x7fd4c80b2f80 2024-09-21T02:03:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 -- 172.21.15.88:0/229097348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80b2f80 msgr2=0x7fd4c80c2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 --2- 172.21.15.88:0/229097348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80b2f80 0x7fd4c80c2030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000b9e0 tx=0x7fd4c000bea0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 -- 172.21.15.88:0/229097348 shutdown_connections 2024-09-21T02:03:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 --2- 172.21.15.88:0/229097348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c80b2f80 0x7fd4c80c2030 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 --2- 172.21.15.88:0/229097348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4c8070950 0x7fd4c81f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 --2- 172.21.15.88:0/229097348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4c8070540 0x7fd4c815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 -- 172.21.15.88:0/229097348 >> 172.21.15.88:0/229097348 conn(0x7fd4c806d390 msgr2=0x7fd4c815a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 -- 172.21.15.88:0/229097348 shutdown_connections 2024-09-21T02:03:33.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.693+0000 7fd4b4ff9640 1 -- 172.21.15.88:0/229097348 wait complete. 2024-09-21T02:03:33.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.698+0000 7f310a7fc640 1 -- 172.21.15.88:0/3174784991 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f3104030960 con 0x7f310c15c370 2024-09-21T02:03:33.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.698+0000 7f27e7fff640 1 -- 172.21.15.88:0/1054366902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f27dc005580 con 0x7f27fc0cd000 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310c15c370 msgr2=0x7f310c0c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310c15c370 0x7f310c0c6860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f310402af50 tx=0x7f3104003ec0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 shutdown_connections 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f310c153e30 0x7f310c14f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f310c153310 0x7f310c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 --2- 172.21.15.88:0/3174784991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310c15c370 0x7f310c0c6860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 >> 172.21.15.88:0/3174784991 conn(0x7f310c06d390 msgr2=0x7f310c0c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 shutdown_connections 2024-09-21T02:03:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.700+0000 7f3113a7e640 1 -- 172.21.15.88:0/3174784991 wait complete. 2024-09-21T02:03:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.704+0000 7f1257fff640 1 -- 172.21.15.88:0/849926326 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v21)=0 v21) ==== 116+0+0 (secure 0 0 0) 0x7f1264016930 con 0x7f12700c6810 2024-09-21T02:03:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12700c6810 msgr2=0x7f127015d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 --2- 172.21.15.88:0/849926326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12700c6810 0x7f127015d0b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f126400d990 tx=0x7f126400de50 comp rx=0 tx=0).stop 2024-09-21T02:03:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 shutdown_connections 2024-09-21T02:03:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 --2- 172.21.15.88:0/849926326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12700c6810 0x7f127015d0b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 --2- 172.21.15.88:0/849926326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12700c7560 0x7f12701ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 --2- 172.21.15.88:0/849926326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1270070950 0x7f12702029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 >> 172.21.15.88:0/849926326 conn(0x7f127006d390 msgr2=0x7f12700c94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 shutdown_connections 2024-09-21T02:03:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.705+0000 7f1275e07640 1 -- 172.21.15.88:0/849926326 wait complete. 2024-09-21T02:03:33.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.707+0000 7f85b27fc640 1 -- 172.21.15.88:0/388709758 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f85a8005850 con 0x7f85c00469d0 2024-09-21T02:03:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.710+0000 7f22eaffd640 1 -- 172.21.15.88:0/1690461570 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v22)=0 v22) ==== 119+0+0 (secure 0 0 0) 0x7f22e4026cc0 con 0x7f22f4152840 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.711+0000 7f22e8ff9640 1 -- 172.21.15.88:0/1690461570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f4152840 msgr2=0x7f22f41f5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.711+0000 7f22e8ff9640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f4152840 0x7f22f41f5fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c430 tx=0x7f22e400c8f0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.712+0000 7f22e8ff9640 1 -- 172.21.15.88:0/1690461570 shutdown_connections 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.712+0000 7f22e8ff9640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f415c510 0x7f22f415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.712+0000 7f22e8ff9640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f4152840 0x7f22f41f5fc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.712+0000 7f22e8ff9640 1 --2- 172.21.15.88:0/1690461570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f4074bd0 0x7f22f40c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.712+0000 7f22e8ff9640 1 -- 172.21.15.88:0/1690461570 >> 172.21.15.88:0/1690461570 conn(0x7f22f406d8f0 msgr2=0x7f22f4157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.712+0000 7f22e8ff9640 1 -- 172.21.15.88:0/1690461570 shutdown_connections 2024-09-21T02:03:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.712+0000 7f22e8ff9640 1 -- 172.21.15.88:0/1690461570 wait complete. 2024-09-21T02:03:33.721 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.721 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.721 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.722 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.722 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.720+0000 7f212bfff640 1 -- 172.21.15.88:0/1423554986 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v23)=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7f2138009c70 con 0x7f214015bb90 2024-09-21T02:03:33.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.720+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f63e00dc5f0 con 0x7f63e017d4e0 2024-09-21T02:03:33.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.721+0000 7f2129ffb640 1 -- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f214015bb90 msgr2=0x7f21400c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.721+0000 7f2129ffb640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f214015bb90 0x7f21400c9cf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f213800b160 tx=0x7f2138008d20 comp rx=0 tx=0).stop 2024-09-21T02:03:33.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.721+0000 7f2129ffb640 1 -- 172.21.15.88:0/1423554986 shutdown_connections 2024-09-21T02:03:33.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.721+0000 7f2129ffb640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f214015bb90 0x7f21400c9cf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.721+0000 7f2129ffb640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21400c2220 0x7f21400c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.721+0000 7f2129ffb640 1 --2- 172.21.15.88:0/1423554986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21400af860 0x7f21400b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.721+0000 7f2129ffb640 1 -- 172.21.15.88:0/1423554986 >> 172.21.15.88:0/1423554986 conn(0x7f214006cd60 msgr2=0x7f214006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.722+0000 7f2129ffb640 1 -- 172.21.15.88:0/1423554986 shutdown_connections 2024-09-21T02:03:33.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.722+0000 7f2129ffb640 1 -- 172.21.15.88:0/1423554986 wait complete. 2024-09-21T02:03:33.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_dir_prefetch', 'True'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.727 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.726+0000 7f53bbfff640 1 -- 172.21.15.88:0/2008303010 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v24)=0 v24) ==== 108+0+0 (secure 0 0 0) 0x7f53cc004a40 con 0x7f53d40cd030 2024-09-21T02:03:33.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 -- 172.21.15.88:0/2008303010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd030 msgr2=0x7f53d4167f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 --2- 172.21.15.88:0/2008303010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd030 0x7f53d4167f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00ca50 tx=0x7f53cc00cf10 comp rx=0 tx=0).stop 2024-09-21T02:03:33.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 -- 172.21.15.88:0/2008303010 shutdown_connections 2024-09-21T02:03:33.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 --2- 172.21.15.88:0/2008303010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53d40c4300 0x7f53d40c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 --2- 172.21.15.88:0/2008303010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53d40cd030 0x7f53d4167f10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 --2- 172.21.15.88:0/2008303010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53d4073d10 0x7f53d4159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 -- 172.21.15.88:0/2008303010 >> 172.21.15.88:0/2008303010 conn(0x7f53d406cf70 msgr2=0x7f53d4076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 -- 172.21.15.88:0/2008303010 shutdown_connections 2024-09-21T02:03:33.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.727+0000 7f53b9ffb640 1 -- 172.21.15.88:0/2008303010 wait complete. 2024-09-21T02:03:33.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.730+0000 7f015affd640 1 -- 172.21.15.88:0/2935976789 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f0160011770 con 0x7f0164072bf0 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.731+0000 7f0158ff9640 1 -- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0164072bf0 msgr2=0x7f0164159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.731+0000 7f0158ff9640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0164072bf0 0x7f0164159f70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f016000c370 tx=0x7f016000c830 comp rx=0 tx=0).stop 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.732+0000 7f0158ff9640 1 -- 172.21.15.88:0/2935976789 shutdown_connections 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.732+0000 7f0158ff9640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01640c6990 0x7f0164157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.732+0000 7f0158ff9640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0164073b40 0x7f016415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.732+0000 7f0158ff9640 1 --2- 172.21.15.88:0/2935976789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0164072bf0 0x7f0164159f70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.732+0000 7f0158ff9640 1 -- 172.21.15.88:0/2935976789 >> 172.21.15.88:0/2935976789 conn(0x7f016406cf70 msgr2=0x7f01640c7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.732+0000 7f0158ff9640 1 -- 172.21.15.88:0/2935976789 shutdown_connections 2024-09-21T02:03:33.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.732+0000 7f0158ff9640 1 -- 172.21.15.88:0/2935976789 wait complete. 2024-09-21T02:03:33.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.734+0000 7f27f27fc640 1 -- 172.21.15.88:0/1054366902 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v26)=0 v26) ==== 111+0+0 (secure 0 0 0) 0x7f27ec032440 con 0x7f27fc0cd000 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 -- 172.21.15.88:0/1054366902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27fc0cd000 msgr2=0x7f27fc153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 --2- 172.21.15.88:0/1054366902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27fc0cd000 0x7f27fc153bf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27ec02aa50 tx=0x7f27ec030570 comp rx=0 tx=0).stop 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 -- 172.21.15.88:0/1054366902 shutdown_connections 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 --2- 172.21.15.88:0/1054366902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27fc0cd410 0x7f27fc0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 --2- 172.21.15.88:0/1054366902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27fc0cd000 0x7f27fc153bf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 --2- 172.21.15.88:0/1054366902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27fc072bf0 0x7f27fc1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 -- 172.21.15.88:0/1054366902 >> 172.21.15.88:0/1054366902 conn(0x7f27fc06cf70 msgr2=0x7f27fc15bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 -- 172.21.15.88:0/1054366902 shutdown_connections 2024-09-21T02:03:33.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.735+0000 7f27e7fff640 1 -- 172.21.15.88:0/1054366902 wait complete. 2024-09-21T02:03:33.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.738+0000 7f85c4ff9640 1 -- 172.21.15.88:0/388709758 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v27)=0 v27) ==== 106+0+0 (secure 0 0 0) 0x7f85b400a0d0 con 0x7f85c00469d0 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 -- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c00469d0 msgr2=0x7f85c01776e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c00469d0 0x7f85c01776e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85b4007440 tx=0x7f85b4003820 comp rx=0 tx=0).stop 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 -- 172.21.15.88:0/388709758 shutdown_connections 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85c00d16a0 0x7f85c0005b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c00469d0 0x7f85c01776e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 --2- 172.21.15.88:0/388709758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85c0044cb0 0x7f85c00cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 -- 172.21.15.88:0/388709758 >> 172.21.15.88:0/388709758 conn(0x7f85c004fe30 msgr2=0x7f85c00da030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 -- 172.21.15.88:0/388709758 shutdown_connections 2024-09-21T02:03:33.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.740+0000 7f85b27fc640 1 -- 172.21.15.88:0/388709758 wait complete. 2024-09-21T02:03:33.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.744+0000 7f63ddffb640 1 -- 172.21.15.88:0/182852062 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f63d8003cf0 con 0x7f63e017d4e0 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.745+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e017d4e0 msgr2=0x7f63e00cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.745+0000 7f63ef3c6640 1 --2- 172.21.15.88:0/182852062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e017d4e0 0x7f63e00cfcc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63d800b5b0 tx=0x7f63d800df90 comp rx=0 tx=0).stop 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.746+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 shutdown_connections 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.746+0000 7f63ef3c6640 1 --2- 172.21.15.88:0/182852062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63e00d0200 0x7f63e0048d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.746+0000 7f63ef3c6640 1 --2- 172.21.15.88:0/182852062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e017d4e0 0x7f63e00cfcc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.746+0000 7f63ef3c6640 1 --2- 172.21.15.88:0/182852062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63e00cea90 0x7f63e017cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.746+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 >> 172.21.15.88:0/182852062 conn(0x7f63e004fe60 msgr2=0x7f63e004eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.746+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 shutdown_connections 2024-09-21T02:03:33.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.746+0000 7f63ef3c6640 1 -- 172.21.15.88:0/182852062 wait complete. 2024-09-21T02:03:33.751 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.751 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.756 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.756 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.756 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi088') 2024-09-21T02:03:33.758 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config dump 2024-09-21T02:03:33.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.908+0000 7fa97d8be640 1 -- 172.21.15.88:0/1829512650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 msgr2=0x7fa9780c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.908+0000 7fa97d8be640 1 --2- 172.21.15.88:0/1829512650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa9780c84b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9780c9c30 tx=0x7fa96c02a710 comp rx=0 tx=0).stop 2024-09-21T02:03:33.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.908+0000 7fa97d8be640 1 -- 172.21.15.88:0/1829512650 shutdown_connections 2024-09-21T02:03:33.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.908+0000 7fa97d8be640 1 --2- 172.21.15.88:0/1829512650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa9780c84b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.908+0000 7fa97d8be640 1 -- 172.21.15.88:0/1829512650 >> 172.21.15.88:0/1829512650 conn(0x7fa9780694f0 msgr2=0x7fa978072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.908+0000 7fa97d8be640 1 -- 172.21.15.88:0/1829512650 shutdown_connections 2024-09-21T02:03:33.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.908+0000 7fa97d8be640 1 -- 172.21.15.88:0/1829512650 wait complete. 2024-09-21T02:03:33.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.910+0000 7fa97d8be640 1 Processor -- start 2024-09-21T02:03:33.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.910+0000 7fa97d8be640 1 -- start start 2024-09-21T02:03:33.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa97d8be640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9780c4030 0x7fa978153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa97d8be640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa978150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa97d8be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9780cbaa0 0x7fa97814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:33.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa97d8be640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780ca2c0 con 0x7fa9780c8b60 2024-09-21T02:03:33.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa97d8be640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780c9fe0 con 0x7fa9780c4030 2024-09-21T02:03:33.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa97d8be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa9780c4440 con 0x7fa9780cbaa0 2024-09-21T02:03:33.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa978150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa978150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41340/0 (socket says 172.21.15.88:41340) 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 -- 172.21.15.88:0/2209887865 learned_addr learned my addr 172.21.15.88:0/2209887865 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa976ffd640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9780c4030 0x7fa978153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9777fe640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9780cbaa0 0x7fa97814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 -- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9780cbaa0 msgr2=0x7fa97814b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9780cbaa0 0x7fa97814b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 -- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9780c4030 msgr2=0x7fa978153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9780c4030 0x7fa978153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.911+0000 7fa9767fc640 1 -- 172.21.15.88:0/2209887865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa978153c70 con 0x7fa9780c8b60 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa9767fc640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa978150210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c460 tx=0x7fa96800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa95ffff640 1 -- 172.21.15.88:0/2209887865 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968017070 con 0x7fa9780c8b60 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa95ffff640 1 -- 172.21.15.88:0/2209887865 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9680053c0 con 0x7fa9780c8b60 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9781512c0 con 0x7fa9780c8b60 2024-09-21T02:03:33.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa95ffff640 1 -- 172.21.15.88:0/2209887865 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968003850 con 0x7fa9780c8b60 2024-09-21T02:03:33.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa978150d10 con 0x7fa9780c8b60 2024-09-21T02:03:33.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa95ffff640 1 -- 172.21.15.88:0/2209887865 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa96801a070 con 0x7fa9780c8b60 2024-09-21T02:03:33.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.912+0000 7fa95ffff640 1 -- 172.21.15.88:0/2209887865 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa96801c780 con 0x7fa9780c8b60 2024-09-21T02:03:33.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.913+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa978159720 con 0x7fa9780c8b60 2024-09-21T02:03:33.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.915+0000 7fa95ffff640 1 -- 172.21.15.88:0/2209887865 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa968021080 con 0x7fa9780c8b60 2024-09-21T02:03:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.995+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa9781533d0 con 0x7fa9780c8b60 2024-09-21T02:03:33.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.995+0000 7fa95ffff640 1 -- 172.21.15.88:0/2209887865 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v28) ==== 59+0+1848 (secure 0 0 0) 0x7fa968011d40 con 0x7fa9780c8b60 2024-09-21T02:03:33.996 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:03:33.996 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:03:33.996 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:03:33.996 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_bal_split_size 100 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_dir_prefetch true 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:03:33.997 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:03:33.998 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 msgr2=0x7fa978150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa978150210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c460 tx=0x7fa96800c920 comp rx=0 tx=0).stop 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 shutdown_connections 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9780cbaa0 0x7fa97814b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780c8b60 0x7fa978150210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 --2- 172.21.15.88:0/2209887865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9780c4030 0x7fa978153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 >> 172.21.15.88:0/2209887865 conn(0x7fa9780694f0 msgr2=0x7fa9780718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 shutdown_connections 2024-09-21T02:03:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:33.996+0000 7fa97d8be640 1 -- 172.21.15.88:0/2209887865 wait complete. 2024-09-21T02:03:34.008 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T02:03:34.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T02:03:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.118+0000 7f25694b6640 1 -- 172.21.15.88:0/3202762362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c8e60 msgr2=0x7f256415a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.118+0000 7f25694b6640 1 --2- 172.21.15.88:0/3202762362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c8e60 0x7f256415a2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f255c005fd0 tx=0x7f255c02a9d0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.118+0000 7f25694b6640 1 -- 172.21.15.88:0/3202762362 shutdown_connections 2024-09-21T02:03:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.118+0000 7f25694b6640 1 --2- 172.21.15.88:0/3202762362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c8e60 0x7f256415a2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.118+0000 7f25694b6640 1 -- 172.21.15.88:0/3202762362 >> 172.21.15.88:0/3202762362 conn(0x7f2564068d00 msgr2=0x7f2564069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.118+0000 7f25694b6640 1 -- 172.21.15.88:0/3202762362 shutdown_connections 2024-09-21T02:03:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.118+0000 7f25694b6640 1 -- 172.21.15.88:0/3202762362 wait complete. 2024-09-21T02:03:34.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.120+0000 7f25694b6640 1 Processor -- start 2024-09-21T02:03:34.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.120+0000 7f25694b6640 1 -- start start 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25694b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c5d90 0x7f25641f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25694b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25640c8e60 0x7f25640c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25694b6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25640c9930 0x7f2564149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25694b6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2564153cb0 con 0x7f25640c9930 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25694b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2564159dc0 con 0x7f25640c5d90 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25694b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2564153e30 con 0x7f25640c8e60 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25637fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25640c9930 0x7f2564149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25637fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25640c9930 0x7f2564149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41352/0 (socket says 172.21.15.88:41352) 2024-09-21T02:03:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f25637fe640 1 -- 172.21.15.88:0/2451303419 learned_addr learned my addr 172.21.15.88:0/2451303419 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f2562ffd640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c5d90 0x7f25641f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.121+0000 7f255bfff640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25640c8e60 0x7f25640c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25637fe640 1 -- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25640c8e60 msgr2=0x7f25640c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25637fe640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25640c8e60 0x7f25640c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25637fe640 1 -- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c5d90 msgr2=0x7f25641f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25637fe640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c5d90 0x7f25641f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25637fe640 1 -- 172.21.15.88:0/2451303419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25641f5b60 con 0x7f25640c9930 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25637fe640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25640c9930 0x7f2564149720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f255c02afc0 tx=0x7f255c031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f255b7fe640 1 -- 172.21.15.88:0/2451303419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c045070 con 0x7f25640c9930 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2564156b70 con 0x7f25640c9930 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f255b7fe640 1 -- 172.21.15.88:0/2451303419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f255c02fc70 con 0x7f25640c9930 2024-09-21T02:03:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f255b7fe640 1 -- 172.21.15.88:0/2451303419 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c003bd0 con 0x7f25640c9930 2024-09-21T02:03:34.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25641f5100 con 0x7f25640c9930 2024-09-21T02:03:34.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f255b7fe640 1 -- 172.21.15.88:0/2451303419 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f255c048070 con 0x7f25640c9930 2024-09-21T02:03:34.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.122+0000 7f255b7fe640 1 -- 172.21.15.88:0/2451303419 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f255c031690 con 0x7f25640c9930 2024-09-21T02:03:34.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.123+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2564071f60 con 0x7f25640c9930 2024-09-21T02:03:34.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.127+0000 7f255b7fe640 1 -- 172.21.15.88:0/2451303419 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f255c04f020 con 0x7f25640c9930 2024-09-21T02:03:34.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.190+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f25640c9320 con 0x7f25640c9930 2024-09-21T02:03:34.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.191+0000 7f255b7fe640 1 -- 172.21.15.88:0/2451303419 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f255c033020 con 0x7f25640c9930 2024-09-21T02:03:34.192 INFO:teuthology.orchestra.run.smithi088.stderr:adjusted tunables profile to default 2024-09-21T02:03:34.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25640c9930 msgr2=0x7f2564149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25640c9930 0x7f2564149720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f255c02afc0 tx=0x7f255c031df0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 shutdown_connections 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25640c9930 0x7f2564149720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25640c8e60 0x7f25640c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 --2- 172.21.15.88:0/2451303419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25640c5d90 0x7f25641f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 >> 172.21.15.88:0/2451303419 conn(0x7f2564068d00 msgr2=0x7f2564157560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 shutdown_connections 2024-09-21T02:03:34.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:34.192+0000 7f25694b6640 1 -- 172.21.15.88:0/2451303419 wait complete. 2024-09-21T02:03:34.204 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T02:03:34.204 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T02:03:34.204 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:34.204 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T02:03:34.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:34.228 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T02:03:34.293 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:34.293 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T02:03:34.357 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:03:34.357 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T02:03:34.422 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:34.422 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T02:03:34.448 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:34.448 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T02:03:34.514 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:34.514 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T02:03:34.578 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-21T02:03:34.578 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T02:03:34.643 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 0c94ead8-72ba-41a1-b18a-c9b2c1de32c9 0 2024-09-21T02:03:34.795 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 -- 172.21.15.195:0/3331580399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 msgr2=0x7f14bc152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.795 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/3331580399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc152900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0ce4a0 tx=0x7f14b002a5f0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 -- 172.21.15.195:0/3331580399 shutdown_connections 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/3331580399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14bc0c91d0 0x7f14bc06de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/3331580399 >> v2:172.21.15.88:3300/0 conn(0x7f14bc0c7af0 0x7f14bc0c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/3331580399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc152900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 -- 172.21.15.195:0/3331580399 >> 172.21.15.195:0/3331580399 conn(0x7f14bc0720b0 msgr2=0x7f14bc0cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 -- 172.21.15.195:0/3331580399 shutdown_connections 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.794+0000 7f14c2bc3640 1 -- 172.21.15.195:0/3331580399 wait complete. 2024-09-21T02:03:34.796 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.796+0000 7f14c2bc3640 1 Processor -- start 2024-09-21T02:03:34.797 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.796+0000 7f14c2bc3640 1 -- start start 2024-09-21T02:03:34.799 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c2bc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c2bc3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14bc0c7af0 0x7f14bc157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c2bc3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14bc0c91d0 0x7f14bc1484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c2bc3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc159190 con 0x7f14bc0c7af0 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c2bc3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc159480 con 0x7f14bc0c2e60 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c2bc3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f14bc06f1f0 con 0x7f14bc0c91d0 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c095f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c095f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc14acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:59562/0 (socket says 172.21.15.195:59562) 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c095f640 1 -- 172.21.15.195:0/1680687423 learned_addr learned my addr 172.21.15.195:0/1680687423 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.797+0000 7f14c1160640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14bc0c91d0 0x7f14bc1484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14bbfff640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14bc0c7af0 0x7f14bc157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c095f640 1 -- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14bc0c91d0 msgr2=0x7f14bc1484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.800 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c095f640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14bc0c91d0 0x7f14bc1484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c095f640 1 -- 172.21.15.195:0/1680687423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14bc0c7af0 msgr2=0x7f14bc157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c095f640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14bc0c7af0 0x7f14bc157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c095f640 1 -- 172.21.15.195:0/1680687423 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc149470 con 0x7f14bc0c2e60 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c095f640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc14acc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14b0034040 tx=0x7f14b0003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14b97fa640 1 -- 172.21.15.195:0/1680687423 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0035040 con 0x7f14bc0c2e60 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14b97fa640 1 -- 172.21.15.195:0/1680687423 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b0003e20 con 0x7f14bc0c2e60 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14b97fa640 1 -- 172.21.15.195:0/1680687423 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00049c0 con 0x7f14bc0c2e60 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc149f30 con 0x7f14bc0c2e60 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc14b370 con 0x7f14bc0c2e60 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14b97fa640 1 -- 172.21.15.195:0/1680687423 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f14b0004ef0 con 0x7f14bc0c2e60 2024-09-21T02:03:34.801 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.798+0000 7f14b97fa640 1 -- 172.21.15.195:0/1680687423 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f14b004b780 con 0x7f14bc0c2e60 2024-09-21T02:03:34.802 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.799+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc14b5d0 con 0x7f14bc0c2e60 2024-09-21T02:03:34.802 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.801+0000 7f14b97fa640 1 -- 172.21.15.195:0/1680687423 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f14b0038d80 con 0x7f14bc0c2e60 2024-09-21T02:03:34.872 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.872+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0c94ead8-72ba-41a1-b18a-c9b2c1de32c9", "id": 0} v 0) -- 0x7f14bc14ecc0 con 0x7f14bc0c2e60 2024-09-21T02:03:34.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.876+0000 7f14b97fa640 1 -- 172.21.15.195:0/1680687423 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c94ead8-72ba-41a1-b18a-c9b2c1de32c9", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f14b004b9d0 con 0x7f14bc0c2e60 2024-09-21T02:03:34.876 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-09-21T02:03:34.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 msgr2=0x7f14bc14acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc14acc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14b0034040 tx=0x7f14b0003820 comp rx=0 tx=0).stop 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 shutdown_connections 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14bc0c91d0 0x7f14bc1484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14bc0c7af0 0x7f14bc157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 --2- 172.21.15.195:0/1680687423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14bc0c2e60 0x7f14bc14acc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 >> 172.21.15.195:0/1680687423 conn(0x7f14bc0720b0 msgr2=0x7f14bc073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 shutdown_connections 2024-09-21T02:03:34.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:34.877+0000 7f14c2bc3640 1 -- 172.21.15.195:0/1680687423 wait complete. 2024-09-21T02:03:34.888 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new dd02029f-64ef-4c29-b948-68face049bd9 1 2024-09-21T02:03:35.002 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 -- 172.21.15.195:0/3634059248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c7af0 msgr2=0x7fd11c0c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.002 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 --2- 172.21.15.195:0/3634059248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c7af0 0x7fd11c0c8c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd10c005fd0 tx=0x7fd10c02a670 comp rx=0 tx=0).stop 2024-09-21T02:03:35.002 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 -- 172.21.15.195:0/3634059248 shutdown_connections 2024-09-21T02:03:35.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 --2- 172.21.15.195:0/3634059248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c7af0 0x7fd11c0c8c90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 --2- 172.21.15.195:0/3634059248 >> v2:172.21.15.88:3300/0 conn(0x7fd11c0c2e60 0x7fd11c152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 -- 172.21.15.195:0/3634059248 >> 172.21.15.195:0/3634059248 conn(0x7fd11c0720b0 msgr2=0x7fd11c0cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 -- 172.21.15.195:0/3634059248 shutdown_connections 2024-09-21T02:03:35.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.001+0000 7fd121f94640 1 -- 172.21.15.195:0/3634059248 wait complete. 2024-09-21T02:03:35.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.003+0000 7fd121f94640 1 Processor -- start 2024-09-21T02:03:35.003 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.003+0000 7fd121f94640 1 -- start start 2024-09-21T02:03:35.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd121f94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c0c2e60 0x7fd11c15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd121f94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd11c0c7af0 0x7fd11c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd121f94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c91d0 0x7fd11c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd121f94640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c159190 con 0x7fd11c0c2e60 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd121f94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c159480 con 0x7fd11c0c7af0 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd121f94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd11c06f1f0 con 0x7fd11c0c91d0 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd11bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c91d0 0x7fd11c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd11affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd11c0c7af0 0x7fd11c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd11bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c91d0 0x7fd11c1539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:36886/0 (socket says 172.21.15.195:36886) 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd11affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd11c0c7af0 0x7fd11c14ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:59572/0 (socket says 172.21.15.195:59572) 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd11bfff640 1 -- 172.21.15.195:0/4018360687 learned_addr learned my addr 172.21.15.195:0/4018360687 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.004+0000 7fd11b7fe640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c0c2e60 0x7fd11c15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd11bfff640 1 -- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd11c0c7af0 msgr2=0x7fd11c14ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd11bfff640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd11c0c7af0 0x7fd11c14ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd11bfff640 1 -- 172.21.15.195:0/4018360687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c0c2e60 msgr2=0x7fd11c15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd11bfff640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c0c2e60 0x7fd11c15b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd11bfff640 1 -- 172.21.15.195:0/4018360687 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c15b940 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd11affd640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd11c0c7af0 0x7fd11c14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd11bfff640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c91d0 0x7fd11c1539e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd114002230 tx=0x7fd11400cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd103fff640 1 -- 172.21.15.195:0/4018360687 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114003b70 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd103fff640 1 -- 172.21.15.195:0/4018360687 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd114003d10 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c14d6b0 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd103fff640 1 -- 172.21.15.195:0/4018360687 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114005330 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c1e1d90 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd103fff640 1 -- 172.21.15.195:0/4018360687 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd1140056e0 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.005+0000 7fd103fff640 1 -- 172.21.15.195:0/4018360687 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd1140127b0 con 0x7fd11c0c91d0 2024-09-21T02:03:35.008 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.006+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11c152f20 con 0x7fd11c0c91d0 2024-09-21T02:03:35.009 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.008+0000 7fd103fff640 1 -- 172.21.15.195:0/4018360687 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd11c0c42e0 con 0x7fd11c0c91d0 2024-09-21T02:03:35.079 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.079+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd02029f-64ef-4c29-b948-68face049bd9", "id": 1} v 0) -- 0x7fd11c1e1b60 con 0x7fd11c0c91d0 2024-09-21T02:03:35.083 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.082+0000 7fd103fff640 1 -- 172.21.15.195:0/4018360687 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd02029f-64ef-4c29-b948-68face049bd9", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd114012b30 con 0x7fd11c0c91d0 2024-09-21T02:03:35.083 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-09-21T02:03:35.084 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.083+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c91d0 msgr2=0x7fd11c1539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.084 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.083+0000 7fd121f94640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c91d0 0x7fd11c1539e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd114002230 tx=0x7fd11400cb90 comp rx=0 tx=0).stop 2024-09-21T02:03:35.084 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.083+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 shutdown_connections 2024-09-21T02:03:35.085 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.083+0000 7fd121f94640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd11c0c91d0 0x7fd11c1539e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.085 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.083+0000 7fd121f94640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd11c0c7af0 0x7fd11c14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.085 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.083+0000 7fd121f94640 1 --2- 172.21.15.195:0/4018360687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c0c2e60 0x7fd11c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.085 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.083+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 >> 172.21.15.195:0/4018360687 conn(0x7fd11c0720b0 msgr2=0x7fd11c150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.085 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.084+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 shutdown_connections 2024-09-21T02:03:35.085 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.084+0000 7fd121f94640 1 -- 172.21.15.195:0/4018360687 wait complete. 2024-09-21T02:03:35.095 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new f0c10054-48b6-4b37-ac91-1501da5ef01f 2 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 -- 172.21.15.195:0/1841751242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d80c9300 msgr2=0x7f18d8152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 --2- 172.21.15.195:0/1841751242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d80c9300 0x7f18d8152470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005fd0 tx=0x7f18d402a670 comp rx=0 tx=0).stop 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 -- 172.21.15.195:0/1841751242 shutdown_connections 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 --2- 172.21.15.195:0/1841751242 >> v2:172.21.15.88:3300/0 conn(0x7f18d81529b0 0x7f18d8158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 --2- 172.21.15.195:0/1841751242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d80c9300 0x7f18d8152470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 -- 172.21.15.195:0/1841751242 >> 172.21.15.195:0/1841751242 conn(0x7f18d8071570 msgr2=0x7f18d8071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 -- 172.21.15.195:0/1841751242 shutdown_connections 2024-09-21T02:03:35.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.207+0000 7f18e0739640 1 -- 172.21.15.195:0/1841751242 wait complete. 2024-09-21T02:03:35.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.209+0000 7f18e0739640 1 Processor -- start 2024-09-21T02:03:35.209 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.209+0000 7f18e0739640 1 -- start start 2024-09-21T02:03:35.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18e0739640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d80c6eb0 0x7f18d8156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.212 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18e0739640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d80c9300 0x7f18d8149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18e0739640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d81529b0 0x7f18d81f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18e0739640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f18d80c9b30 con 0x7f18d80c6eb0 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18e0739640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18d80753f0 con 0x7f18d80c9300 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18e0739640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f18d80ce080 con 0x7f18d81529b0 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18ddcd4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d80c9300 0x7f18d8149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d81529b0 0x7f18d81f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d81529b0 0x7f18d81f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:36900/0 (socket says 172.21.15.195:36900) 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18ddcd4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d80c9300 0x7f18d8149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:59588/0 (socket says 172.21.15.195:59588) 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 -- 172.21.15.195:0/173689945 learned_addr learned my addr 172.21.15.195:0/173689945 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18de4d5640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d80c6eb0 0x7f18d8156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 -- 172.21.15.195:0/173689945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d80c9300 msgr2=0x7f18d8149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d80c9300 0x7f18d8149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 -- 172.21.15.195:0/173689945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d80c6eb0 msgr2=0x7f18d8156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d80c6eb0 0x7f18d8156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.210+0000 7f18decd6640 1 -- 172.21.15.195:0/173689945 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d81ef480 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18decd6640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d81529b0 0x7f18d81f1d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18d000ea70 tx=0x7f18d000ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18caffd640 1 -- 172.21.15.195:0/173689945 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d000cde0 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18caffd640 1 -- 172.21.15.195:0/173689945 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18d0004c10 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d81f22d0 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18caffd640 1 -- 172.21.15.195:0/173689945 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0005020 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d81eeb50 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18caffd640 1 -- 172.21.15.195:0/173689945 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f18d0005550 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.211+0000 7f18caffd640 1 -- 172.21.15.195:0/173689945 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f18d0010850 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.212+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d8152470 con 0x7f18d81529b0 2024-09-21T02:03:35.214 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.214+0000 7f18caffd640 1 -- 172.21.15.195:0/173689945 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f18d81ef480 con 0x7f18d81529b0 2024-09-21T02:03:35.285 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.284+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f0c10054-48b6-4b37-ac91-1501da5ef01f", "id": 2} v 0) -- 0x7f18d8073b70 con 0x7f18d81529b0 2024-09-21T02:03:35.289 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.288+0000 7f18caffd640 1 -- 172.21.15.195:0/173689945 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0c10054-48b6-4b37-ac91-1501da5ef01f", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f18d00059d0 con 0x7f18d81529b0 2024-09-21T02:03:35.289 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-09-21T02:03:35.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d81529b0 msgr2=0x7f18d81f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d81529b0 0x7f18d81f1d90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18d000ea70 tx=0x7f18d000ef30 comp rx=0 tx=0).stop 2024-09-21T02:03:35.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 shutdown_connections 2024-09-21T02:03:35.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18d81529b0 0x7f18d81f1d90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.290 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d80c9300 0x7f18d8149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 --2- 172.21.15.195:0/173689945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18d80c6eb0 0x7f18d8156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 >> 172.21.15.195:0/173689945 conn(0x7f18d8071570 msgr2=0x7f18d8156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 shutdown_connections 2024-09-21T02:03:35.291 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.289+0000 7f18e0739640 1 -- 172.21.15.195:0/173689945 wait complete. 2024-09-21T02:03:35.300 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 2f19c3d6-3682-430a-9c0c-9bbe0d92d338 3 2024-09-21T02:03:35.412 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 -- 172.21.15.195:0/1601442713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 msgr2=0x7f9b400ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.412 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 --2- 172.21.15.195:0/1601442713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 0x7f9b400ca880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b38005fd0 tx=0x7f9b3802a820 comp rx=0 tx=0).stop 2024-09-21T02:03:35.412 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 -- 172.21.15.195:0/1601442713 shutdown_connections 2024-09-21T02:03:35.412 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 --2- 172.21.15.195:0/1601442713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 0x7f9b400ca880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.412 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 --2- 172.21.15.195:0/1601442713 >> v2:172.21.15.195:3300/0 conn(0x7f9b40157ed0 0x7f9b400c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.412 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 -- 172.21.15.195:0/1601442713 >> 172.21.15.195:0/1601442713 conn(0x7f9b40071570 msgr2=0x7f9b40071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.413 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 -- 172.21.15.195:0/1601442713 shutdown_connections 2024-09-21T02:03:35.413 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.411+0000 7f9b4549b640 1 -- 172.21.15.195:0/1601442713 wait complete. 2024-09-21T02:03:35.414 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.413+0000 7f9b4549b640 1 Processor -- start 2024-09-21T02:03:35.414 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.413+0000 7f9b4549b640 1 -- start start 2024-09-21T02:03:35.416 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b4549b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 0x7f9b401ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.416 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b4549b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b4014e7d0 0x7f9b400c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.416 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b4549b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b40157ed0 0x7f9b400cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.416 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b4549b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40152480 con 0x7f9b4014e7d0 2024-09-21T02:03:35.416 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b4549b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9b400c9f50 con 0x7f9b40157ed0 2024-09-21T02:03:35.416 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b4549b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4014ebe0 con 0x7f9b400ca470 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3effd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 0x7f9b401ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b40157ed0 0x7f9b400cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3effd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 0x7f9b401ee410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:36922/0 (socket says 172.21.15.195:36922) 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3effd640 1 -- 172.21.15.195:0/1191378008 learned_addr learned my addr 172.21.15.195:0/1191378008 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3e7fc640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b4014e7d0 0x7f9b400c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3f7fe640 1 -- 172.21.15.195:0/1191378008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 msgr2=0x7f9b401ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3f7fe640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 0x7f9b401ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.414+0000 7f9b3f7fe640 1 -- 172.21.15.195:0/1191378008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b4014e7d0 msgr2=0x7f9b400c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b3f7fe640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b4014e7d0 0x7f9b400c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b3f7fe640 1 -- 172.21.15.195:0/1191378008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b401f1070 con 0x7f9b40157ed0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b3f7fe640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b40157ed0 0x7f9b400cc860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009000 tx=0x7f9b38032c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b277fe640 1 -- 172.21.15.195:0/1191378008 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38037040 con 0x7f9b40157ed0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b277fe640 1 -- 172.21.15.195:0/1191378008 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b38042070 con 0x7f9b40157ed0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b401ee950 con 0x7f9b40157ed0 2024-09-21T02:03:35.417 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b277fe640 1 -- 172.21.15.195:0/1191378008 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380042a0 con 0x7f9b40157ed0 2024-09-21T02:03:35.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b40073b90 con 0x7f9b40157ed0 2024-09-21T02:03:35.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b277fe640 1 -- 172.21.15.195:0/1191378008 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b38004440 con 0x7f9b40157ed0 2024-09-21T02:03:35.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.415+0000 7f9b277fe640 1 -- 172.21.15.195:0/1191378008 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f9b38047d50 con 0x7f9b40157ed0 2024-09-21T02:03:35.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.416+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b400c6ff0 con 0x7f9b40157ed0 2024-09-21T02:03:35.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.418+0000 7f9b277fe640 1 -- 172.21.15.195:0/1191378008 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9b400c6ff0 con 0x7f9b40157ed0 2024-09-21T02:03:35.488 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.488+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2f19c3d6-3682-430a-9c0c-9bbe0d92d338", "id": 3} v 0) -- 0x7f9b400c8ae0 con 0x7f9b40157ed0 2024-09-21T02:03:35.492 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.492+0000 7f9b277fe640 1 -- 172.21.15.195:0/1191378008 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f19c3d6-3682-430a-9c0c-9bbe0d92d338", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f9b38047460 con 0x7f9b40157ed0 2024-09-21T02:03:35.492 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b40157ed0 msgr2=0x7f9b400cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b40157ed0 0x7f9b400cc860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009000 tx=0x7f9b38032c90 comp rx=0 tx=0).stop 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 shutdown_connections 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b40157ed0 0x7f9b400cc860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b4014e7d0 0x7f9b400c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 --2- 172.21.15.195:0/1191378008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b400ca470 0x7f9b401ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 >> 172.21.15.195:0/1191378008 conn(0x7f9b40071570 msgr2=0x7f9b40155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 shutdown_connections 2024-09-21T02:03:35.494 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.493+0000 7f9b4549b640 1 -- 172.21.15.195:0/1191378008 wait complete. 2024-09-21T02:03:35.504 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 5e972a73-17cc-4107-9991-07c6ef899ddf 4 2024-09-21T02:03:35.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.617+0000 7f43f844d640 1 -- 172.21.15.195:0/1587040643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c2e60 msgr2=0x7f43f0152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.618 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.617+0000 7f43f844d640 1 --2- 172.21.15.195:0/1587040643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c2e60 0x7f43f0152900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009830 tx=0x7f43e402a640 comp rx=0 tx=0).stop 2024-09-21T02:03:35.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.617+0000 7f43f844d640 1 -- 172.21.15.195:0/1587040643 shutdown_connections 2024-09-21T02:03:35.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.617+0000 7f43f844d640 1 --2- 172.21.15.195:0/1587040643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c2e60 0x7f43f0152900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.617+0000 7f43f844d640 1 -- 172.21.15.195:0/1587040643 >> 172.21.15.195:0/1587040643 conn(0x7f43f00720b0 msgr2=0x7f43f00cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.617+0000 7f43f844d640 1 -- 172.21.15.195:0/1587040643 shutdown_connections 2024-09-21T02:03:35.619 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.617+0000 7f43f844d640 1 -- 172.21.15.195:0/1587040643 wait complete. 2024-09-21T02:03:35.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.619+0000 7f43f844d640 1 Processor -- start 2024-09-21T02:03:35.620 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.619+0000 7f43f844d640 1 -- start start 2024-09-21T02:03:35.622 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f844d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43f006de70 0x7f43f0155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f844d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f00c2e60 0x7f43f006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f844d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c7af0 0x7f43f014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f844d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0159190 con 0x7f43f006de70 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f844d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f43f00c4930 con 0x7f43f00c7af0 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f844d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f43f006f1f0 con 0x7f43f00c2e60 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f69ea640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c7af0 0x7f43f014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.620+0000 7f43f59e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f00c2e60 0x7f43f006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c7af0 0x7f43f014d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:59618/0 (socket says 172.21.15.195:59618) 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 -- 172.21.15.195:0/4036017409 learned_addr learned my addr 172.21.15.195:0/4036017409 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f59e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f00c2e60 0x7f43f006cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:36926/0 (socket says 172.21.15.195:36926) 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f61e9640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43f006de70 0x7f43f0155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 -- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f00c2e60 msgr2=0x7f43f006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.623 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f00c2e60 0x7f43f006cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 -- 172.21.15.195:0/4036017409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43f006de70 msgr2=0x7f43f0155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43f006de70 0x7f43f0155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 -- 172.21.15.195:0/4036017409 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f0154680 con 0x7f43f00c7af0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f59e8640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f00c2e60 0x7f43f006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f69ea640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c7af0 0x7f43f014d760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43e800cb10 tx=0x7f43e800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43e2ffd640 1 -- 172.21.15.195:0/4036017409 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8016020 con 0x7f43f00c7af0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f01ef8d0 con 0x7f43f00c7af0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43e2ffd640 1 -- 172.21.15.195:0/4036017409 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43e800ad00 con 0x7f43f00c7af0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43e2ffd640 1 -- 172.21.15.195:0/4036017409 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8012680 con 0x7f43f00c7af0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01efe90 con 0x7f43f00c7af0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.621+0000 7f43e2ffd640 1 -- 172.21.15.195:0/4036017409 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f43e8019020 con 0x7f43f00c7af0 2024-09-21T02:03:35.624 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.622+0000 7f43e2ffd640 1 -- 172.21.15.195:0/4036017409 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f43e801bd10 con 0x7f43f00c7af0 2024-09-21T02:03:35.625 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.622+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f0151060 con 0x7f43f00c7af0 2024-09-21T02:03:35.625 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.624+0000 7f43e2ffd640 1 -- 172.21.15.195:0/4036017409 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f43f0151060 con 0x7f43f00c7af0 2024-09-21T02:03:35.696 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.695+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5e972a73-17cc-4107-9991-07c6ef899ddf", "id": 4} v 0) -- 0x7f43f0152900 con 0x7f43f00c7af0 2024-09-21T02:03:35.700 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.699+0000 7f43e2ffd640 1 -- 172.21.15.195:0/4036017409 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e972a73-17cc-4107-9991-07c6ef899ddf", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f43e8020020 con 0x7f43f00c7af0 2024-09-21T02:03:35.700 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c7af0 msgr2=0x7f43f014d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c7af0 0x7f43f014d760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43e800cb10 tx=0x7f43e800a060 comp rx=0 tx=0).stop 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 shutdown_connections 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f00c7af0 0x7f43f014d760 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f00c2e60 0x7f43f006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 --2- 172.21.15.195:0/4036017409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43f006de70 0x7f43f0155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 >> 172.21.15.195:0/4036017409 conn(0x7f43f00720b0 msgr2=0x7f43f0076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 shutdown_connections 2024-09-21T02:03:35.701 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.700+0000 7f43f844d640 1 -- 172.21.15.195:0/4036017409 wait complete. 2024-09-21T02:03:35.711 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f 5 2024-09-21T02:03:35.824 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.823+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2275614963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb5806de70 msgr2=0x7fdb5806e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.823+0000 7fdb5e3ed640 1 --2- 172.21.15.195:0/2275614963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb5806de70 0x7fdb5806e280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdb50005fd0 tx=0x7fdb5002a940 comp rx=0 tx=0).stop 2024-09-21T02:03:35.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.824+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2275614963 shutdown_connections 2024-09-21T02:03:35.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.824+0000 7fdb5e3ed640 1 --2- 172.21.15.195:0/2275614963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb5806de70 0x7fdb5806e280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.824+0000 7fdb5e3ed640 1 --2- 172.21.15.195:0/2275614963 >> v2:172.21.15.195:3300/0 conn(0x7fdb580c2e60 0x7fdb58152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.824+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2275614963 >> 172.21.15.195:0/2275614963 conn(0x7fdb580720b0 msgr2=0x7fdb580cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.824+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2275614963 shutdown_connections 2024-09-21T02:03:35.825 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.824+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2275614963 wait complete. 2024-09-21T02:03:35.826 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.825+0000 7fdb5e3ed640 1 Processor -- start 2024-09-21T02:03:35.826 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.825+0000 7fdb5e3ed640 1 -- start start 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5e3ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb5806de70 0x7fdb581558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5e3ed640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb580c2e60 0x7fdb581f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5e3ed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb580c7af0 0x7fdb58074910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5e3ed640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58159190 con 0x7fdb580c2e60 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5e3ed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58159480 con 0x7fdb5806de70 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5e3ed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5806f1f0 con 0x7fdb580c7af0 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb5806de70 0x7fdb581558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5c98a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb580c7af0 0x7fdb58074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb5806de70 0x7fdb581558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:59634/0 (socket says 172.21.15.195:59634) 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 -- 172.21.15.195:0/2239047316 learned_addr learned my addr 172.21.15.195:0/2239047316 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:35.829 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb577fe640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb580c2e60 0x7fdb581f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 -- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb580c7af0 msgr2=0x7fdb58074910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb580c7af0 0x7fdb58074910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 -- 172.21.15.195:0/2239047316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb580c2e60 msgr2=0x7fdb581f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb580c2e60 0x7fdb581f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 -- 172.21.15.195:0/2239047316 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb58155e10 con 0x7fdb5806de70 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb577fe640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb580c2e60 0x7fdb581f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb5c98a640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb580c7af0 0x7fdb58074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb57fff640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb5806de70 0x7fdb581558d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000e9c0 tx=0x7fdb4000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.827+0000 7fdb54ff9640 1 -- 172.21.15.195:0/2239047316 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000cd20 con 0x7fdb5806de70 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.828+0000 7fdb54ff9640 1 -- 172.21.15.195:0/2239047316 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb40004020 con 0x7fdb5806de70 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.828+0000 7fdb54ff9640 1 -- 172.21.15.195:0/2239047316 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40004450 con 0x7fdb5806de70 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.828+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb580b2c50 con 0x7fdb5806de70 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.828+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb580c5490 con 0x7fdb5806de70 2024-09-21T02:03:35.830 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.828+0000 7fdb54ff9640 1 -- 172.21.15.195:0/2239047316 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb40003980 con 0x7fdb5806de70 2024-09-21T02:03:35.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.828+0000 7fdb54ff9640 1 -- 172.21.15.195:0/2239047316 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fdb400189b0 con 0x7fdb5806de70 2024-09-21T02:03:35.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.829+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb58155fe0 con 0x7fdb5806de70 2024-09-21T02:03:35.831 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.831+0000 7fdb54ff9640 1 -- 172.21.15.195:0/2239047316 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fdb40038860 con 0x7fdb5806de70 2024-09-21T02:03:35.900 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.899+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f", "id": 5} v 0) -- 0x7fdb58152f20 con 0x7fdb5806de70 2024-09-21T02:03:35.904 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.903+0000 7fdb54ff9640 1 -- 172.21.15.195:0/2239047316 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fdb40026020 con 0x7fdb5806de70 2024-09-21T02:03:35.904 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb5806de70 msgr2=0x7fdb581558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb5806de70 0x7fdb581558d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000e9c0 tx=0x7fdb4000ee80 comp rx=0 tx=0).stop 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 shutdown_connections 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb580c7af0 0x7fdb58074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb580c2e60 0x7fdb581f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 --2- 172.21.15.195:0/2239047316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb5806de70 0x7fdb581558d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 >> 172.21.15.195:0/2239047316 conn(0x7fdb580720b0 msgr2=0x7fdb58150110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 shutdown_connections 2024-09-21T02:03:35.905 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:35.904+0000 7fdb5e3ed640 1 -- 172.21.15.195:0/2239047316 wait complete. 2024-09-21T02:03:35.915 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new f048a1c4-2584-43fd-b35d-cc9f5297f892 6 2024-09-21T02:03:36.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 -- 172.21.15.195:0/3817358757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 msgr2=0x7ff6140c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 --2- 172.21.15.195:0/3817358757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 0x7ff6140c8c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009830 tx=0x7ff5fc02a5a0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 -- 172.21.15.195:0/3817358757 shutdown_connections 2024-09-21T02:03:36.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 --2- 172.21.15.195:0/3817358757 >> v2:172.21.15.195:3300/0 conn(0x7ff6140c91d0 0x7ff61406de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 --2- 172.21.15.195:0/3817358757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 0x7ff6140c8c90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.028 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 -- 172.21.15.195:0/3817358757 >> 172.21.15.195:0/3817358757 conn(0x7ff6140720b0 msgr2=0x7ff6140cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:36.029 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 -- 172.21.15.195:0/3817358757 shutdown_connections 2024-09-21T02:03:36.029 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.027+0000 7ff618e4d640 1 -- 172.21.15.195:0/3817358757 wait complete. 2024-09-21T02:03:36.029 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.029+0000 7ff618e4d640 1 Processor -- start 2024-09-21T02:03:36.029 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.029+0000 7ff618e4d640 1 -- start start 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff618e4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6140c2e60 0x7ff6141f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff618e4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 0x7ff614150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff618e4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6140c91d0 0x7ff61414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff618e4d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff614159190 con 0x7ff6140c91d0 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff618e4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff614159480 con 0x7ff6140c2e60 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff618e4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff61406f1f0 con 0x7ff6140c7af0 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff61259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6140c2e60 0x7ff6141f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff61259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6140c2e60 0x7ff6141f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:59640/0 (socket says 172.21.15.195:59640) 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff611d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 0x7ff614150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff61259c640 1 -- 172.21.15.195:0/1013985898 learned_addr learned my addr 172.21.15.195:0/1013985898 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.030+0000 7ff612d9d640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6140c91d0 0x7ff61414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff611d9b640 1 -- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6140c2e60 msgr2=0x7ff6141f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff611d9b640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6140c2e60 0x7ff6141f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff611d9b640 1 -- 172.21.15.195:0/1013985898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6140c91d0 msgr2=0x7ff61414d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff611d9b640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6140c91d0 0x7ff61414d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff611d9b640 1 -- 172.21.15.195:0/1013985898 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6141f5880 con 0x7ff6140c7af0 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff61259c640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6140c2e60 0x7ff6141f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff612d9d640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6140c91d0 0x7ff61414d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff611d9b640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 0x7ff614150b50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc031000 tx=0x7ff5fc033ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff60affd640 1 -- 172.21.15.195:0/1013985898 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc031b50 con 0x7ff6140c7af0 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff60affd640 1 -- 172.21.15.195:0/1013985898 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5fc031e00 con 0x7ff6140c7af0 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6140c6d20 con 0x7ff6140c7af0 2024-09-21T02:03:36.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff60affd640 1 -- 172.21.15.195:0/1013985898 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc033330 con 0x7ff6140c7af0 2024-09-21T02:03:36.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6140ccc70 con 0x7ff6140c7af0 2024-09-21T02:03:36.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff60affd640 1 -- 172.21.15.195:0/1013985898 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff5fc0334d0 con 0x7ff6140c7af0 2024-09-21T02:03:36.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.031+0000 7ff60affd640 1 -- 172.21.15.195:0/1013985898 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7ff5fc003d90 con 0x7ff6140c7af0 2024-09-21T02:03:36.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.032+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6140c8c90 con 0x7ff6140c7af0 2024-09-21T02:03:36.035 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.034+0000 7ff60affd640 1 -- 172.21.15.195:0/1013985898 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff5fc03f030 con 0x7ff6140c7af0 2024-09-21T02:03:36.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.124+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f048a1c4-2584-43fd-b35d-cc9f5297f892", "id": 6} v 0) -- 0x7ff614074df0 con 0x7ff6140c7af0 2024-09-21T02:03:36.128 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.128+0000 7ff60affd640 1 -- 172.21.15.195:0/1013985898 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f048a1c4-2584-43fd-b35d-cc9f5297f892", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7ff5fc0035f0 con 0x7ff6140c7af0 2024-09-21T02:03:36.128 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 msgr2=0x7ff614150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 0x7ff614150b50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc031000 tx=0x7ff5fc033ba0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 shutdown_connections 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6140c91d0 0x7ff61414d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6140c7af0 0x7ff614150b50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 --2- 172.21.15.195:0/1013985898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6140c2e60 0x7ff6141f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 >> 172.21.15.195:0/1013985898 conn(0x7ff6140720b0 msgr2=0x7ff614074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 shutdown_connections 2024-09-21T02:03:36.130 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.129+0000 7ff618e4d640 1 -- 172.21.15.195:0/1013985898 wait complete. 2024-09-21T02:03:36.140 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 8dc8088b-3941-476b-8d8b-e946018d3842 7 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 -- 172.21.15.195:0/1978287363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 msgr2=0x7f0e0c0c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 --2- 172.21.15.195:0/1978287363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c0c9300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005fd0 tx=0x7f0dfc02a670 comp rx=0 tx=0).stop 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 -- 172.21.15.195:0/1978287363 shutdown_connections 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 --2- 172.21.15.195:0/1978287363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c0c9300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 --2- 172.21.15.195:0/1978287363 >> v2:172.21.15.88:3300/0 conn(0x7f0e0c159460 0x7f0e0c0c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 -- 172.21.15.195:0/1978287363 >> 172.21.15.195:0/1978287363 conn(0x7f0e0c071570 msgr2=0x7f0e0c071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 -- 172.21.15.195:0/1978287363 shutdown_connections 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.251+0000 7f0e12920640 1 -- 172.21.15.195:0/1978287363 wait complete. 2024-09-21T02:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.253+0000 7f0e12920640 1 Processor -- start 2024-09-21T02:03:36.254 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.253+0000 7f0e12920640 1 -- start start 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.254+0000 7f0e12920640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c1f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.254+0000 7f0e12920640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0c152550 0x7f0e0c153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.254+0000 7f0e12920640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e0c1482e0 0x7f0e0c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.254+0000 7f0e12920640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0c8390 con 0x7f0e0c152550 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.254+0000 7f0e12920640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c158f40 con 0x7f0e0c1482e0 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.254+0000 7f0e12920640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c0c9be0 con 0x7f0e0c0c6f40 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e0bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c1f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e10ebd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e0c1482e0 0x7f0e0c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e0bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c1f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:36972/0 (socket says 172.21.15.195:36972) 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e0bfff640 1 -- 172.21.15.195:0/1858094804 learned_addr learned my addr 172.21.15.195:0/1858094804 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-09-21T02:03:36.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e0b7fe640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0c152550 0x7f0e0c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e10ebd640 1 -- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 msgr2=0x7f0e0c1f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e10ebd640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c1f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e10ebd640 1 -- 172.21.15.195:0/1858094804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0c152550 msgr2=0x7f0e0c153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e10ebd640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0c152550 0x7f0e0c153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e10ebd640 1 -- 172.21.15.195:0/1858094804 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1f3660 con 0x7f0e0c1482e0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e0bfff640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c1f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e10ebd640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e0c1482e0 0x7f0e0c1478c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c700 tx=0x7f0e0400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e08ff9640 1 -- 172.21.15.195:0/1858094804 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04003b70 con 0x7f0e0c1482e0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e08ff9640 1 -- 172.21.15.195:0/1858094804 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e04003d10 con 0x7f0e0c1482e0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c14ac40 con 0x7f0e0c1482e0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e08ff9640 1 -- 172.21.15.195:0/1858094804 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04005330 con 0x7f0e0c1482e0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.255+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1493e0 con 0x7f0e0c1482e0 2024-09-21T02:03:36.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.256+0000 7f0e08ff9640 1 -- 172.21.15.195:0/1858094804 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e040054d0 con 0x7f0e0c1482e0 2024-09-21T02:03:36.259 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.256+0000 7f0e08ff9640 1 -- 172.21.15.195:0/1858094804 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f0e04012ba0 con 0x7f0e0c1482e0 2024-09-21T02:03:36.259 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.256+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c0c9920 con 0x7f0e0c1482e0 2024-09-21T02:03:36.259 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.258+0000 7f0e08ff9640 1 -- 172.21.15.195:0/1858094804 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0e04012500 con 0x7f0e0c1482e0 2024-09-21T02:03:36.329 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.328+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8dc8088b-3941-476b-8d8b-e946018d3842", "id": 7} v 0) -- 0x7f0e0c14f620 con 0x7f0e0c1482e0 2024-09-21T02:03:36.333 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.332+0000 7f0e08ff9640 1 -- 172.21.15.195:0/1858094804 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8dc8088b-3941-476b-8d8b-e946018d3842", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f0e040126e0 con 0x7f0e0c1482e0 2024-09-21T02:03:36.333 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-09-21T02:03:36.334 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e0c1482e0 msgr2=0x7f0e0c1478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.334 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e0c1482e0 0x7f0e0c1478c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c700 tx=0x7f0e0400cbc0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.334 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 shutdown_connections 2024-09-21T02:03:36.334 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e0c1482e0 0x7f0e0c1478c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.334 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0c152550 0x7f0e0c153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 --2- 172.21.15.195:0/1858094804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e0c0c6f40 0x7f0e0c1f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 >> 172.21.15.195:0/1858094804 conn(0x7f0e0c071570 msgr2=0x7f0e0c0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:36.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 shutdown_connections 2024-09-21T02:03:36.335 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-21T02:03:36.333+0000 7f0e12920640 1 -- 172.21.15.195:0/1858094804 wait complete. 2024-09-21T02:03:36.345 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T02:03:36.345 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T02:03:36.347 INFO:tasks.ceph.osd.0:Started 2024-09-21T02:03:36.347 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T02:03:36.347 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T02:03:36.349 INFO:tasks.ceph.osd.1:Started 2024-09-21T02:03:36.349 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T02:03:36.349 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T02:03:36.351 INFO:tasks.ceph.osd.2:Started 2024-09-21T02:03:36.352 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T02:03:36.352 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T02:03:36.353 INFO:tasks.ceph.osd.3:Started 2024-09-21T02:03:36.354 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T02:03:36.354 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T02:03:36.356 INFO:tasks.ceph.osd.4:Started 2024-09-21T02:03:36.356 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T02:03:36.356 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T02:03:36.358 INFO:tasks.ceph.osd.5:Started 2024-09-21T02:03:36.358 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T02:03:36.358 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T02:03:36.360 INFO:tasks.ceph.osd.6:Started 2024-09-21T02:03:36.360 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T02:03:36.360 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T02:03:36.362 INFO:tasks.ceph.osd.7:Started 2024-09-21T02:03:36.362 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:03:36.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 -- 172.21.15.88:0/1086222958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80755d0 msgr2=0x7f2be8151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 --2- 172.21.15.88:0/1086222958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80755d0 0x7f2be8151d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009830 tx=0x7f2bd802a5a0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 -- 172.21.15.88:0/1086222958 shutdown_connections 2024-09-21T02:03:36.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 --2- 172.21.15.88:0/1086222958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be80c9800 0x7f2be815b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 --2- 172.21.15.88:0/1086222958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80755d0 0x7f2be8151d60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 -- 172.21.15.88:0/1086222958 >> 172.21.15.88:0/1086222958 conn(0x7f2be80c9f90 msgr2=0x7f2be80ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:36.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 -- 172.21.15.88:0/1086222958 shutdown_connections 2024-09-21T02:03:36.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.502+0000 7f2bee352640 1 -- 172.21.15.88:0/1086222958 wait complete. 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.504+0000 7f2bee352640 1 Processor -- start 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.504+0000 7f2bee352640 1 -- start start 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bee352640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be80755d0 0x7f2be81476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bee352640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80c9800 0x7f2be81541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bee352640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be815bb40 0x7f2be81f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bee352640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2be815bf50 con 0x7f2be80755d0 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bee352640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81589c0 con 0x7f2be80c9800 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bee352640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2be80c9c10 con 0x7f2be815bb40 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be80755d0 0x7f2be81476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2be7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be80755d0 0x7f2be81476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41402/0 (socket says 172.21.15.88:41402) 2024-09-21T02:03:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2be7fff640 1 -- 172.21.15.88:0/1475096013 learned_addr learned my addr 172.21.15.88:0/1475096013 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bdffff640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80c9800 0x7f2be81541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bec8ef640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be815bb40 0x7f2be81f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bdffff640 1 -- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be815bb40 msgr2=0x7f2be81f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bdffff640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be815bb40 0x7f2be81f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bdffff640 1 -- 172.21.15.88:0/1475096013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be80755d0 msgr2=0x7f2be81476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bdffff640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be80755d0 0x7f2be81476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2bdffff640 1 -- 172.21.15.88:0/1475096013 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be8147c30 con 0x7f2be80c9800 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.505+0000 7f2be7fff640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be80755d0 0x7f2be81476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2bdffff640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80c9800 0x7f2be81541c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e6a0 tx=0x7f2bd000eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2be57fa640 1 -- 172.21.15.88:0/1475096013 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000cab0 con 0x7f2be80c9800 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2be57fa640 1 -- 172.21.15.88:0/1475096013 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bd000cd40 con 0x7f2be80c9800 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be8154df0 con 0x7f2be80c9800 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2be57fa640 1 -- 172.21.15.88:0/1475096013 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0010690 con 0x7f2be80c9800 2024-09-21T02:03:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be8153b20 con 0x7f2be80c9800 2024-09-21T02:03:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2be57fa640 1 -- 172.21.15.88:0/1475096013 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2bd0010830 con 0x7f2be80c9800 2024-09-21T02:03:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.506+0000 7f2be57fa640 1 -- 172.21.15.88:0/1475096013 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2bd0010b20 con 0x7f2be80c9800 2024-09-21T02:03:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.507+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be8154b60 con 0x7f2be80c9800 2024-09-21T02:03:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.509+0000 7f2be57fa640 1 -- 172.21.15.88:0/1475096013 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2bd0018dc0 con 0x7f2be80c9800 2024-09-21T02:03:36.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.570+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2be8155650 con 0x7f2be80c9800 2024-09-21T02:03:36.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.570+0000 7f2be57fa640 1 -- 172.21.15.88:0/1475096013 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2bd0010d20 con 0x7f2be80c9800 2024-09-21T02:03:36.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:03:36.571 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":10,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:36.330786+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:36.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80c9800 msgr2=0x7f2be81541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80c9800 0x7f2be81541c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e6a0 tx=0x7f2bd000eb60 comp rx=0 tx=0).stop 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 shutdown_connections 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be815bb40 0x7f2be81f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be80c9800 0x7f2be81541c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 --2- 172.21.15.88:0/1475096013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be80755d0 0x7f2be81476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 >> 172.21.15.88:0/1475096013 conn(0x7f2be80c9f90 msgr2=0x7f2be80c7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 shutdown_connections 2024-09-21T02:03:36.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:36.571+0000 7f2bee352640 1 -- 172.21.15.88:0/1475096013 wait complete. 2024-09-21T02:03:36.583 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T02:03:36.583 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T02:03:37.389 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.494+0000 7f2504630640 1 -- 172.21.15.88:0/1385954389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 msgr2=0x7f24fc159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.494+0000 7f2504630640 1 --2- 172.21.15.88:0/1385954389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc159d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005fd0 tx=0x7f24f802a670 comp rx=0 tx=0).stop 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.495+0000 7f2504630640 1 -- 172.21.15.88:0/1385954389 shutdown_connections 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.495+0000 7f2504630640 1 --2- 172.21.15.88:0/1385954389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24fc1621f0 0x7f24fc164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.495+0000 7f2504630640 1 --2- 172.21.15.88:0/1385954389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc159d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.495+0000 7f2504630640 1 -- 172.21.15.88:0/1385954389 >> 172.21.15.88:0/1385954389 conn(0x7f24fc071880 msgr2=0x7f24fc071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.495+0000 7f2504630640 1 -- 172.21.15.88:0/1385954389 shutdown_connections 2024-09-21T02:03:37.496 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.495+0000 7f2504630640 1 -- 172.21.15.88:0/1385954389 wait complete. 2024-09-21T02:03:37.497 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.496+0000 7f2504630640 1 Processor -- start 2024-09-21T02:03:37.497 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.496+0000 7f2504630640 1 -- start start 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.497+0000 7f2504630640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24fc153c50 0x7f24fc15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.497+0000 7f2504630640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc1f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2504630640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24fc1621f0 0x7f24fc0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2504630640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc15a270 con 0x7f24fc159920 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2504630640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc154060 con 0x7f24fc1621f0 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2504630640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc0c8290 con 0x7f24fc153c50 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc1f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc1f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41422/0 (socket says 172.21.15.88:41422) 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 -- 172.21.15.88:0/674064166 learned_addr learned my addr 172.21.15.88:0/674064166 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2502bcd640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24fc1621f0 0x7f24fc0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f25023cc640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24fc153c50 0x7f24fc15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 -- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24fc153c50 msgr2=0x7f24fc15f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24fc153c50 0x7f24fc15f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.500 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 -- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24fc1621f0 msgr2=0x7f24fc0c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24fc1621f0 0x7f24fc0c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 -- 172.21.15.88:0/674064166 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc15f9a0 con 0x7f24fc159920 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f25023cc640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24fc153c50 0x7f24fc15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.498+0000 7f2501bcb640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc1f5aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24f8034040 tx=0x7f24f8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.499+0000 7f24eeffd640 1 -- 172.21.15.88:0/674064166 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f803e030 con 0x7f24fc159920 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.499+0000 7f24eeffd640 1 -- 172.21.15.88:0/674064166 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24f8003e00 con 0x7f24fc159920 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.499+0000 7f2504630640 1 -- 172.21.15.88:0/674064166 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc1f5fe0 con 0x7f24fc159920 2024-09-21T02:03:37.501 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.499+0000 7f24eeffd640 1 -- 172.21.15.88:0/674064166 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8035040 con 0x7f24fc159920 2024-09-21T02:03:37.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.499+0000 7f2504630640 1 -- 172.21.15.88:0/674064166 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc0c4a00 con 0x7f24fc159920 2024-09-21T02:03:37.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.500+0000 7f24ecff9640 1 -- 172.21.15.88:0/674064166 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d8005290 con 0x7f24fc159920 2024-09-21T02:03:37.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.500+0000 7f24eeffd640 1 -- 172.21.15.88:0/674064166 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f24f8041030 con 0x7f24fc159920 2024-09-21T02:03:37.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.501+0000 7f24eeffd640 1 -- 172.21.15.88:0/674064166 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f24f8073060 con 0x7f24fc159920 2024-09-21T02:03:37.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.502+0000 7f24eeffd640 1 -- 172.21.15.88:0/674064166 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f24f8004f20 con 0x7f24fc159920 2024-09-21T02:03:37.563 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.563+0000 7f24ecff9640 1 -- 172.21.15.88:0/674064166 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24d8005850 con 0x7f24fc159920 2024-09-21T02:03:37.564 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.563+0000 7f24eeffd640 1 -- 172.21.15.88:0/674064166 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f24f803acf0 con 0x7f24fc159920 2024-09-21T02:03:37.564 INFO:teuthology.misc.health.smithi088.stdout: 2024-09-21T02:03:37.564 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":10,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:36.330786+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:37.565 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 -- 172.21.15.88:0/674064166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 msgr2=0x7f24fc1f5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:37.565 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc1f5aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24f8034040 tx=0x7f24f8003910 comp rx=0 tx=0).stop 2024-09-21T02:03:37.565 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 -- 172.21.15.88:0/674064166 shutdown_connections 2024-09-21T02:03:37.565 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24fc1621f0 0x7f24fc0c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.565 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24fc159920 0x7f24fc1f5aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.565 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 --2- 172.21.15.88:0/674064166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24fc153c50 0x7f24fc15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:37.566 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 -- 172.21.15.88:0/674064166 >> 172.21.15.88:0/674064166 conn(0x7f24fc071880 msgr2=0x7f24fc152f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:37.566 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 -- 172.21.15.88:0/674064166 shutdown_connections 2024-09-21T02:03:37.566 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:37.564+0000 7f24ecff9640 1 -- 172.21.15.88:0/674064166 wait complete. 2024-09-21T02:03:37.575 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T02:03:37.590 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T02:03:37.590+0000 7fd431219640 -1 Falling back to public interface 2024-09-21T02:03:37.824 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-21T02:03:37.824+0000 7f564c97d640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T02:03:37.825 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-21T02:03:37.825+0000 7f564c97d640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T02:03:37.833 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T02:03:37.832+0000 7f3ace6dc640 -1 Falling back to public interface 2024-09-21T02:03:37.838 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T02:03:37.838+0000 7f72de61c640 -1 Falling back to public interface 2024-09-21T02:03:37.852 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T02:03:37.851+0000 7fa59c902640 -1 Falling back to public interface 2024-09-21T02:03:37.868 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T02:03:37.867+0000 7f7732042640 -1 Falling back to public interface 2024-09-21T02:03:37.881 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T02:03:37.880+0000 7f766700f640 -1 Falling back to public interface 2024-09-21T02:03:38.100 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T02:03:38.099+0000 7f910f9db640 -1 Falling back to public interface 2024-09-21T02:03:38.329 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T02:03:38.328+0000 7f8647145640 -1 Falling back to public interface 2024-09-21T02:03:41.356 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T02:03:41.355+0000 7f3ace6dc640 -1 osd.0 0 log_to_monitors true 2024-09-21T02:03:41.398 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T02:03:41.398+0000 7fd431219640 -1 osd.6 0 log_to_monitors true 2024-09-21T02:03:41.675 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T02:03:41.675+0000 7f7732042640 -1 osd.4 0 log_to_monitors true 2024-09-21T02:03:41.828 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T02:03:41.828+0000 7f72de61c640 -1 osd.7 0 log_to_monitors true 2024-09-21T02:03:41.890 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T02:03:41.889+0000 7fa59c902640 -1 osd.2 0 log_to_monitors true 2024-09-21T02:03:42.106 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T02:03:42.105+0000 7f910f9db640 -1 osd.3 0 log_to_monitors true 2024-09-21T02:03:42.167 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T02:03:42.167+0000 7f766700f640 -1 osd.5 0 log_to_monitors true 2024-09-21T02:03:42.342 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T02:03:42.341+0000 7f8647145640 -1 osd.1 0 log_to_monitors true 2024-09-21T02:03:43.270 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T02:03:43.270+0000 7fd42cb25640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T02:03:43.282 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-09-21T02:03:43.282+0000 7fd42491b640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:43.439 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T02:03:43.438+0000 7f3aca66a640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T02:03:43.461 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T02:03:43.460+0000 7f3ac2460640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:44.382 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:03:44.498 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 -- 172.21.15.88:0/3173352955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 msgr2=0x7f2d340ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.498 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3173352955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d340ca000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d34073e20 tx=0x7f2d2c02aac0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.498 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 -- 172.21.15.88:0/3173352955 shutdown_connections 2024-09-21T02:03:44.498 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3173352955 >> v2:172.21.15.195:3300/0 conn(0x7f2d34155e80 0x7f2d340705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.498 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3173352955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d340ca000 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.498 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 -- 172.21.15.88:0/3173352955 >> 172.21.15.88:0/3173352955 conn(0x7f2d34071880 msgr2=0x7f2d34071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.498 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 -- 172.21.15.88:0/3173352955 shutdown_connections 2024-09-21T02:03:44.499 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.496+0000 7f2d3c590640 1 -- 172.21.15.88:0/3173352955 wait complete. 2024-09-21T02:03:44.499 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.498+0000 7f2d3c590640 1 Processor -- start 2024-09-21T02:03:44.499 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.498+0000 7f2d3c590640 1 -- start start 2024-09-21T02:03:44.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3c590640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d34070b30 0x7f2d34154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3c590640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d34153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3c590640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d34155e80 0x7f2d34157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3c590640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d340759a0 con 0x7f2d340b3fb0 2024-09-21T02:03:44.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3c590640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d340c31b0 con 0x7f2d34070b30 2024-09-21T02:03:44.502 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3c590640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d34072730 con 0x7f2d34155e80 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d34153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d34153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38134/0 (socket says 172.21.15.88:38134) 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 -- 172.21.15.88:0/3200019362 learned_addr learned my addr 172.21.15.88:0/3200019362 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3a32c640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d34070b30 0x7f2d34154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d3ab2d640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d34155e80 0x7f2d34157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 -- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d34155e80 msgr2=0x7f2d34157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d34155e80 0x7f2d34157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 -- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d34070b30 msgr2=0x7f2d34154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d34070b30 0x7f2d34154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.500+0000 7f2d39b2b640 1 -- 172.21.15.88:0/3200019362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d341f06c0 con 0x7f2d340b3fb0 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.501+0000 7f2d39b2b640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d34153780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d700 tx=0x7f2d2800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.503 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.501+0000 7f2d32ffd640 1 -- 172.21.15.88:0/3200019362 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28017070 con 0x7f2d340b3fb0 2024-09-21T02:03:44.504 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.501+0000 7f2d32ffd640 1 -- 172.21.15.88:0/3200019362 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d28005310 con 0x7f2d340b3fb0 2024-09-21T02:03:44.504 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.501+0000 7f2d32ffd640 1 -- 172.21.15.88:0/3200019362 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28009820 con 0x7f2d340b3fb0 2024-09-21T02:03:44.504 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.501+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d341f2ee0 con 0x7f2d340b3fb0 2024-09-21T02:03:44.504 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.501+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d341f34a0 con 0x7f2d340b3fb0 2024-09-21T02:03:44.504 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.502+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d340c9500 con 0x7f2d340b3fb0 2024-09-21T02:03:44.505 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.503+0000 7f2d32ffd640 1 -- 172.21.15.88:0/3200019362 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f2d280099c0 con 0x7f2d340b3fb0 2024-09-21T02:03:44.505 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.503+0000 7f2d32ffd640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d20078b00 0x7f2d2007b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:44.505 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.504+0000 7f2d32ffd640 1 -- 172.21.15.88:0/3200019362 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2791+0+0 (secure 0 0 0) 0x7f2d2809a0a0 con 0x7f2d340b3fb0 2024-09-21T02:03:44.505 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.504+0000 7f2d3a32c640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d20078b00 0x7f2d2007b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:44.505 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.504+0000 7f2d3a32c640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d20078b00 0x7f2d2007b010 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c035000 tx=0x7f2d2c038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:44.509 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.509+0000 7f2d32ffd640 1 -- 172.21.15.88:0/3200019362 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d28062b60 con 0x7f2d340b3fb0 2024-09-21T02:03:44.789 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T02:03:44.787+0000 7f7662885640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T02:03:44.794 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.793+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d34075780 con 0x7f2d340b3fb0 2024-09-21T02:03:44.796 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.795+0000 7f2d32ffd640 1 -- 172.21.15.88:0/3200019362 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8087 (secure 0 0 0) 0x7f2d28067590 con 0x7f2d340b3fb0 2024-09-21T02:03:44.796 INFO:teuthology.misc.health.smithi088.stdout: 2024-09-21T02:03:44.796 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":14,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:44.614315+0000","last_up_change":"2024-09-21T02:03:43.559083+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:44.797 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T02:03:44.796+0000 7f772dfd0640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T02:03:44.800 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d20078b00 msgr2=0x7f2d2007b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.800 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d20078b00 0x7f2d2007b010 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c035000 tx=0x7f2d2c038040 comp rx=0 tx=0).stop 2024-09-21T02:03:44.800 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 msgr2=0x7f2d34153780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d34153780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d700 tx=0x7f2d2800dbc0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 shutdown_connections 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d20078b00 0x7f2d2007b010 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d34155e80 0x7f2d34157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d340b3fb0 0x7f2d34153780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 --2- 172.21.15.88:0/3200019362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d34070b30 0x7f2d34154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 >> 172.21.15.88:0/3200019362 conn(0x7f2d34071880 msgr2=0x7f2d3414eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 shutdown_connections 2024-09-21T02:03:44.801 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:44.799+0000 7f2d3c590640 1 -- 172.21.15.88:0/3200019362 wait complete. 2024-09-21T02:03:44.801 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-09-21T02:03:44.800+0000 7f765a67b640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:44.808 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-09-21T02:03:44.807+0000 7f7725dc6640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:44.814 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-09-21T02:03:44.832 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T02:03:44.831+0000 7f72da7b5640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T02:03:44.845 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-21T02:03:44.844+0000 7f72d1d96640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:45.269 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T02:03:45.268+0000 7f86438e8640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T02:03:45.277 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T02:03:45.277+0000 7fa598890640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T02:03:45.283 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T02:03:45.282+0000 7f863aec9640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:45.295 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T02:03:45.295+0000 7fa590686640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:45.309 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T02:03:45.309+0000 7f910c17e640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T02:03:45.327 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T02:03:45.326+0000 7f910375f640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:03:51.621 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:03:51.738 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 -- 172.21.15.88:0/2514747674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 msgr2=0x7f0e2c06d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.738 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/2514747674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c06d170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e24005fd0 tx=0x7f0e2402a8b0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.738 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 -- 172.21.15.88:0/2514747674 shutdown_connections 2024-09-21T02:03:51.738 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/2514747674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c06d170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.738 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/2514747674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0c5e20 0x7f0e2c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.739 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 -- 172.21.15.88:0/2514747674 >> 172.21.15.88:0/2514747674 conn(0x7f0e2c0cdb30 msgr2=0x7f0e2c0710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.739 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 -- 172.21.15.88:0/2514747674 shutdown_connections 2024-09-21T02:03:51.739 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.736+0000 7f0e33a0a640 1 -- 172.21.15.88:0/2514747674 wait complete. 2024-09-21T02:03:51.739 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.738+0000 7f0e33a0a640 1 Processor -- start 2024-09-21T02:03:51.739 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.738+0000 7f0e33a0a640 1 -- start start 2024-09-21T02:03:51.742 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e33a0a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.742 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e33a0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0719b0 0x7f0e2c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.742 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e33a0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0c5e20 0x7f0e2c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.742 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e33a0a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c0c72e0 con 0x7f0e2c06cd60 2024-09-21T02:03:51.742 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e33a0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c0c9cf0 con 0x7f0e2c0c5e20 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e33a0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2c071f10 con 0x7f0e2c0719b0 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e317a6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.739+0000 7f0e317a6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c15b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60394/0 (socket says 172.21.15.88:60394) 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e317a6640 1 -- 172.21.15.88:0/4167771279 learned_addr learned my addr 172.21.15.88:0/4167771279 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e30fa5640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0719b0 0x7f0e2c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e317a6640 1 -- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0719b0 msgr2=0x7f0e2c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e317a6640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0719b0 0x7f0e2c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e31fa7640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0c5e20 0x7f0e2c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e317a6640 1 -- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0c5e20 msgr2=0x7f0e2c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e317a6640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0c5e20 0x7f0e2c15ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e317a6640 1 -- 172.21.15.88:0/4167771279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c15dcd0 con 0x7f0e2c06cd60 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e30fa5640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0719b0 0x7f0e2c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e317a6640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c15b660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c3c0 tx=0x7f0e2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.743 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e21ffb640 1 -- 172.21.15.88:0/4167771279 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28017070 con 0x7f0e2c06cd60 2024-09-21T02:03:51.744 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e21ffb640 1 -- 172.21.15.88:0/4167771279 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e280035f0 con 0x7f0e2c06cd60 2024-09-21T02:03:51.744 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c151870 con 0x7f0e2c06cd60 2024-09-21T02:03:51.744 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e21ffb640 1 -- 172.21.15.88:0/4167771279 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28003a80 con 0x7f0e2c06cd60 2024-09-21T02:03:51.744 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.740+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c151e30 con 0x7f0e2c06cd60 2024-09-21T02:03:51.744 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.742+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2c159f80 con 0x7f0e2c06cd60 2024-09-21T02:03:51.745 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.742+0000 7f0e21ffb640 1 -- 172.21.15.88:0/4167771279 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e2801a070 con 0x7f0e2c06cd60 2024-09-21T02:03:51.745 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.743+0000 7f0e21ffb640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0e10078bc0 0x7f0e1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:51.745 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.743+0000 7f0e21ffb640 1 -- 172.21.15.88:0/4167771279 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f0e28099d80 con 0x7f0e2c06cd60 2024-09-21T02:03:51.745 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.743+0000 7f0e30fa5640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0e10078bc0 0x7f0e1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:51.745 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.743+0000 7f0e30fa5640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0e10078bc0 0x7f0e1007b0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0cbfc0 tx=0x7f0e1c005cb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:51.748 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.748+0000 7f0e21ffb640 1 -- 172.21.15.88:0/4167771279 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e2c159f80 con 0x7f0e2c06cd60 2024-09-21T02:03:51.925 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.924+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e2c153e70 con 0x7f0e2c06cd60 2024-09-21T02:03:51.925 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.925+0000 7f0e21ffb640 1 -- 172.21.15.88:0/4167771279 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f0e28062470 con 0x7f0e2c06cd60 2024-09-21T02:03:51.926 INFO:teuthology.misc.health.smithi088.stdout: 2024-09-21T02:03:51.926 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":17,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:47.812170+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:51.928 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0e10078bc0 msgr2=0x7f0e1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.928 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0e10078bc0 0x7f0e1007b0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0cbfc0 tx=0x7f0e1c005cb0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.928 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 msgr2=0x7f0e2c15b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:51.928 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c15b660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c3c0 tx=0x7f0e2800c880 comp rx=0 tx=0).stop 2024-09-21T02:03:51.928 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 shutdown_connections 2024-09-21T02:03:51.928 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0e10078bc0 0x7f0e1007b0d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.928 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e2c0c5e20 0x7f0e2c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.929 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e2c0719b0 0x7f0e2c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.929 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 --2- 172.21.15.88:0/4167771279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e2c06cd60 0x7f0e2c15b660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:51.929 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 >> 172.21.15.88:0/4167771279 conn(0x7f0e2c0cdb30 msgr2=0x7f0e2c0c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:51.929 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 shutdown_connections 2024-09-21T02:03:51.929 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T02:03:51.927+0000 7f0e33a0a640 1 -- 172.21.15.88:0/4167771279 wait complete. 2024-09-21T02:03:51.937 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T02:03:51.938 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T02:03:51.938 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:03:51.938 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:03:51.940 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:03:51.940 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:03:51.940 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:03:51.942 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:03:51.942 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-21T02:03:51.943 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-21T02:03:51.944 INFO:tasks.ceph.mds.e:Started 2024-09-21T02:03:51.945 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:03:51.945 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:03:51.947 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:03:51.948 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:03:51.948 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:03:51.950 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:03:51.950 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-21T02:03:51.950 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-21T02:03:51.952 INFO:tasks.ceph.mds.f:Started 2024-09-21T02:03:51.952 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T02:03:51.952 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:03:52.030 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T02:03:52.030 INFO:tasks.ceph.mds.e.smithi088.stdout:starting mds.e at 2024-09-21T02:03:52.031 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T02:03:52.050 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-09-21T02:03:52.051 INFO:tasks.ceph.mds.f.smithi195.stdout:starting mds.f at 2024-09-21T02:03:52.053 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T02:03:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 -- 172.21.15.88:0/3208320354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f80c99a0 msgr2=0x7fa4f80712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 --2- 172.21.15.88:0/3208320354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f80c99a0 0x7fa4f80712e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8005fd0 tx=0x7fa4e802a5d0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 -- 172.21.15.88:0/3208320354 shutdown_connections 2024-09-21T02:03:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 --2- 172.21.15.88:0/3208320354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4f8153320 0x7fa4f8153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 --2- 172.21.15.88:0/3208320354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f80c99a0 0x7fa4f80712e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 -- 172.21.15.88:0/3208320354 >> 172.21.15.88:0/3208320354 conn(0x7fa4f8073210 msgr2=0x7fa4f80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 -- 172.21.15.88:0/3208320354 shutdown_connections 2024-09-21T02:03:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.074+0000 7fa4fce63640 1 -- 172.21.15.88:0/3208320354 wait complete. 2024-09-21T02:03:52.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.076+0000 7fa4fce63640 1 Processor -- start 2024-09-21T02:03:52.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.076+0000 7fa4fce63640 1 -- start start 2024-09-21T02:03:52.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4fce63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4f80c41a0 0x7fa4f81f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4fce63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4f80c99a0 0x7fa4f8147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4fce63640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f8153320 0x7fa4f81575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4fce63640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8070dc0 con 0x7fa4f8153320 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4fce63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8070ae0 con 0x7fa4f80c99a0 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4fce63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f80c4cc0 con 0x7fa4f80c41a0 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4f6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f8153320 0x7fa4f81575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4f659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4f80c41a0 0x7fa4f81f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4f5d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4f80c99a0 0x7fa4f8147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4f6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f8153320 0x7fa4f81575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60436/0 (socket says 172.21.15.88:60436) 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4f659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4f80c41a0 0x7fa4f81f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:51596/0 (socket says 172.21.15.88:51596) 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.077+0000 7fa4f6d9d640 1 -- 172.21.15.88:0/1073346282 learned_addr learned my addr 172.21.15.88:0/1073346282 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4f6d9d640 1 -- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4f80c41a0 msgr2=0x7fa4f81f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4f6d9d640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4f80c41a0 0x7fa4f81f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4f6d9d640 1 -- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4f80c99a0 msgr2=0x7fa4f8147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4f6d9d640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4f80c99a0 0x7fa4f8147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4f6d9d640 1 -- 172.21.15.88:0/1073346282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f81f4170 con 0x7fa4f8153320 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4f6d9d640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f8153320 0x7fa4f81575b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c730 tx=0x7fa4f000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4e6ffd640 1 -- 172.21.15.88:0/1073346282 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0003b70 con 0x7fa4f8153320 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4e6ffd640 1 -- 172.21.15.88:0/1073346282 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4f0003d10 con 0x7fa4f8153320 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f81590c0 con 0x7fa4f8153320 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4e6ffd640 1 -- 172.21.15.88:0/1073346282 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0005330 con 0x7fa4f8153320 2024-09-21T02:03:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.078+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f80c8e30 con 0x7fa4f8153320 2024-09-21T02:03:52.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.079+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f80c8ba0 con 0x7fa4f8153320 2024-09-21T02:03:52.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.080+0000 7fa4e6ffd640 1 -- 172.21.15.88:0/1073346282 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa4f00054d0 con 0x7fa4f8153320 2024-09-21T02:03:52.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.080+0000 7fa4e6ffd640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa4d8078bc0 0x7fa4d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:52.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.081+0000 7fa4e6ffd640 1 -- 172.21.15.88:0/1073346282 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa4f009a560 con 0x7fa4f8153320 2024-09-21T02:03:52.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.081+0000 7fa4f659c640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa4d8078bc0 0x7fa4d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:52.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.081+0000 7fa4f659c640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa4d8078bc0 0x7fa4d807b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8033000 tx=0x7fa4e8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:52.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.085+0000 7fa4e6ffd640 1 -- 172.21.15.88:0/1073346282 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4f00627c0 con 0x7fa4f8153320 2024-09-21T02:03:52.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.297+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4f80ca8c0 con 0x7fa4f8153320 2024-09-21T02:03:52.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.298+0000 7fa4e6ffd640 1 -- 172.21.15.88:0/1073346282 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fa4f00671f0 con 0x7fa4f8153320 2024-09-21T02:03:52.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:03:52.299 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":2,"btime":"2024-09-21T02:03:52:036233+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:03:52.299 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 2 2024-09-21T02:03:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa4d8078bc0 msgr2=0x7fa4d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa4d8078bc0 0x7fa4d807b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8033000 tx=0x7fa4e8004730 comp rx=0 tx=0).stop 2024-09-21T02:03:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f8153320 msgr2=0x7fa4f81575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f8153320 0x7fa4f81575b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c730 tx=0x7fa4f000cb30 comp rx=0 tx=0).stop 2024-09-21T02:03:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 shutdown_connections 2024-09-21T02:03:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa4d8078bc0 0x7fa4d807b0d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f8153320 0x7fa4f81575b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4f80c99a0 0x7fa4f8147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 --2- 172.21.15.88:0/1073346282 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4f80c41a0 0x7fa4f81f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:52.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 >> 172.21.15.88:0/1073346282 conn(0x7fa4f8073210 msgr2=0x7fa4f8151f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:52.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 shutdown_connections 2024-09-21T02:03:52.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:52.300+0000 7fa4fce63640 1 -- 172.21.15.88:0/1073346282 wait complete. 2024-09-21T02:03:54.312 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:03:54.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.436+0000 7fa54e6f9640 1 -- 172.21.15.88:0/3176642956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa530004d40 msgr2=0x7fa530005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.436+0000 7fa54e6f9640 1 --2- 172.21.15.88:0/3176642956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa530004d40 0x7fa530005150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa540005fd0 tx=0x7fa530005830 comp rx=0 tx=0).stop 2024-09-21T02:03:54.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.437+0000 7fa54cc96640 1 -- 172.21.15.88:0/3176642956 reap_dead start 2024-09-21T02:03:54.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.437+0000 7fa54e6f9640 1 -- 172.21.15.88:0/3176642956 shutdown_connections 2024-09-21T02:03:54.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.437+0000 7fa54e6f9640 1 -- 172.21.15.88:0/3176642956 >> 172.21.15.88:0/3176642956 conn(0x7fa5480729d0 msgr2=0x7fa548072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.437+0000 7fa54e6f9640 1 -- 172.21.15.88:0/3176642956 shutdown_connections 2024-09-21T02:03:54.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.437+0000 7fa54e6f9640 1 -- 172.21.15.88:0/3176642956 wait complete. 2024-09-21T02:03:54.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.438+0000 7fa54e6f9640 1 Processor -- start 2024-09-21T02:03:54.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.439+0000 7fa54e6f9640 1 -- start start 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54e6f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54814ad80 0x7fa548158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54e6f9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa54806e4f0 0x7fa54806e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54e6f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5481556f0 0x7fa548155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54e6f9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa548071920 con 0x7fa54806e4f0 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54e6f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa548070ab0 con 0x7fa54814ad80 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54e6f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa54815a5b0 con 0x7fa5481556f0 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa54806e4f0 0x7fa54806e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa54806e4f0 0x7fa54806e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60446/0 (socket says 172.21.15.88:60446) 2024-09-21T02:03:54.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 -- 172.21.15.88:0/1574815971 learned_addr learned my addr 172.21.15.88:0/1574815971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa547fff640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54814ad80 0x7fa548158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54cc96640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5481556f0 0x7fa548155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 -- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5481556f0 msgr2=0x7fa548155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5481556f0 0x7fa548155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 -- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54814ad80 msgr2=0x7fa548158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54814ad80 0x7fa548158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 -- 172.21.15.88:0/1574815971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5481563e0 con 0x7fa54806e4f0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa547fff640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54814ad80 0x7fa548158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa5477fe640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa54806e4f0 0x7fa54806e900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa53c011ae0 tx=0x7fa53c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa544ff9640 1 -- 172.21.15.88:0/1574815971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c01e070 con 0x7fa54806e4f0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa544ff9640 1 -- 172.21.15.88:0/1574815971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa53c004480 con 0x7fa54806e4f0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5481e2f70 con 0x7fa54806e4f0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.440+0000 7fa544ff9640 1 -- 172.21.15.88:0/1574815971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c00fc60 con 0x7fa54806e4f0 2024-09-21T02:03:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.441+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5480c7810 con 0x7fa54806e4f0 2024-09-21T02:03:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.441+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5480c8f30 con 0x7fa54806e4f0 2024-09-21T02:03:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.442+0000 7fa544ff9640 1 -- 172.21.15.88:0/1574815971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa53c021070 con 0x7fa54806e4f0 2024-09-21T02:03:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.443+0000 7fa544ff9640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa534078b90 0x7fa53407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.443+0000 7fa544ff9640 1 -- 172.21.15.88:0/1574815971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa53c09b3f0 con 0x7fa54806e4f0 2024-09-21T02:03:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.443+0000 7fa547fff640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa534078b90 0x7fa53407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.443+0000 7fa547fff640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa534078b90 0x7fa53407b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa538003670 tx=0x7fa538001ce0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.447+0000 7fa544ff9640 1 -- 172.21.15.88:0/1574815971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa53c063650 con 0x7fa54806e4f0 2024-09-21T02:03:54.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.662+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa548072360 con 0x7fa54806e4f0 2024-09-21T02:03:54.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.662+0000 7fa544ff9640 1 -- 172.21.15.88:0/1574815971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5554 (secure 0 0 0) 0x7fa53c068080 con 0x7fa54806e4f0 2024-09-21T02:03:54.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:03:54.664 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":3,"btime":"2024-09-21T02:03:53:037285+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4398,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T02:03:54.664 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 3 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.664+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa534078b90 msgr2=0x7fa53407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.664+0000 7fa54e6f9640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa534078b90 0x7fa53407b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa538003670 tx=0x7fa538001ce0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.664+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa54806e4f0 msgr2=0x7fa54806e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.664+0000 7fa54e6f9640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa54806e4f0 0x7fa54806e900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa53c011ae0 tx=0x7fa53c009e40 comp rx=0 tx=0).stop 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 shutdown_connections 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa534078b90 0x7fa53407b0a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5481556f0 0x7fa548155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa54806e4f0 0x7fa54806e900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 --2- 172.21.15.88:0/1574815971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54814ad80 0x7fa548158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 >> 172.21.15.88:0/1574815971 conn(0x7fa5480729d0 msgr2=0x7fa54814cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 shutdown_connections 2024-09-21T02:03:54.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.665+0000 7fa54e6f9640 1 -- 172.21.15.88:0/1574815971 wait complete. 2024-09-21T02:03:54.677 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:03:54.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.788+0000 7fe0de3ed640 1 -- 172.21.15.88:0/3272134185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0c00039e0 msgr2=0x7fe0c0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.788+0000 7fe0de3ed640 1 --2- 172.21.15.88:0/3272134185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0c00039e0 0x7fe0c0003e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c290 tx=0x7fe0c0005830 comp rx=0 tx=0).stop 2024-09-21T02:03:54.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.788+0000 7fe0dc98a640 1 -- 172.21.15.88:0/3272134185 reap_dead start 2024-09-21T02:03:54.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.788+0000 7fe0de3ed640 1 -- 172.21.15.88:0/3272134185 shutdown_connections 2024-09-21T02:03:54.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.788+0000 7fe0de3ed640 1 -- 172.21.15.88:0/3272134185 >> 172.21.15.88:0/3272134185 conn(0x7fe0d8073210 msgr2=0x7fe0d80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.788+0000 7fe0de3ed640 1 -- 172.21.15.88:0/3272134185 shutdown_connections 2024-09-21T02:03:54.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.788+0000 7fe0de3ed640 1 -- 172.21.15.88:0/3272134185 wait complete. 2024-09-21T02:03:54.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.790+0000 7fe0de3ed640 1 Processor -- start 2024-09-21T02:03:54.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.790+0000 7fe0de3ed640 1 -- start start 2024-09-21T02:03:54.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0de3ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c00039e0 0x7fe0d80c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0de3ed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0d815b890 0x7fe0d81f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0de3ed640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d815a900 0x7fe0d814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0de3ed640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d8070dc0 con 0x7fe0d815a900 2024-09-21T02:03:54.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0de3ed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d8070ae0 con 0x7fe0c00039e0 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0de3ed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d80c4cc0 con 0x7fe0d815b890 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d815a900 0x7fe0d814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d815a900 0x7fe0d814cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60470/0 (socket says 172.21.15.88:60470) 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 -- 172.21.15.88:0/1837154003 learned_addr learned my addr 172.21.15.88:0/1837154003 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0d7fff640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c00039e0 0x7fe0d80c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0d77fe640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0d815b890 0x7fe0d81f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 -- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0d815b890 msgr2=0x7fe0d81f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0d815b890 0x7fe0d81f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 -- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c00039e0 msgr2=0x7fe0d80c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c00039e0 0x7fe0d80c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.791+0000 7fe0dc98a640 1 -- 172.21.15.88:0/1837154003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d80c8900 con 0x7fe0d815a900 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.792+0000 7fe0dc98a640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d815a900 0x7fe0d814cd00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0004780 tx=0x7fe0d000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.792+0000 7fe0d4ff9640 1 -- 172.21.15.88:0/1837154003 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0004b20 con 0x7fe0d815a900 2024-09-21T02:03:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.792+0000 7fe0d4ff9640 1 -- 172.21.15.88:0/1837154003 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0d0004cc0 con 0x7fe0d815a900 2024-09-21T02:03:54.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.792+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d8155030 con 0x7fe0d815a900 2024-09-21T02:03:54.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.792+0000 7fe0d4ff9640 1 -- 172.21.15.88:0/1837154003 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0003bd0 con 0x7fe0d815a900 2024-09-21T02:03:54.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.792+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d815ae80 con 0x7fe0d815a900 2024-09-21T02:03:54.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.793+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d80c9db0 con 0x7fe0d815a900 2024-09-21T02:03:54.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.794+0000 7fe0d4ff9640 1 -- 172.21.15.88:0/1837154003 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0d0004030 con 0x7fe0d815a900 2024-09-21T02:03:54.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.794+0000 7fe0d4ff9640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe0c4078b90 0x7fe0c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:54.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.794+0000 7fe0d4ff9640 1 -- 172.21.15.88:0/1837154003 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe0d009ada0 con 0x7fe0d815a900 2024-09-21T02:03:54.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.794+0000 7fe0d7fff640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe0c4078b90 0x7fe0c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:54.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.795+0000 7fe0d7fff640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe0c4078b90 0x7fe0c407b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8033000 tx=0x7fe0c8038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:54.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.799+0000 7fe0d4ff9640 1 -- 172.21.15.88:0/1837154003 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0d0062f70 con 0x7fe0d815a900 2024-09-21T02:03:54.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.977+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe0d8152220 con 0x7fe0d815a900 2024-09-21T02:03:54.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.978+0000 7fe0d4ff9640 1 -- 172.21.15.88:0/1837154003 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7fe0d00679a0 con 0x7fe0d815a900 2024-09-21T02:03:54.980 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:03:54.981 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T02:03:54.981 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:03:54.981 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T02:03:54.981 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T02:03:54.981 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:03:54.981 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T02:03:54.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe0c4078b90 msgr2=0x7fe0c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe0c4078b90 0x7fe0c407b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8033000 tx=0x7fe0c8038040 comp rx=0 tx=0).stop 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d815a900 msgr2=0x7fe0d814cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d815a900 0x7fe0d814cd00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0004780 tx=0x7fe0d000f3c0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 shutdown_connections 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe0c4078b90 0x7fe0c407b0a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d815a900 0x7fe0d814cd00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0d815b890 0x7fe0d81f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 --2- 172.21.15.88:0/1837154003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0c00039e0 0x7fe0d80c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 >> 172.21.15.88:0/1837154003 conn(0x7fe0d8073210 msgr2=0x7fe0d814e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 shutdown_connections 2024-09-21T02:03:54.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:54.980+0000 7fe0de3ed640 1 -- 172.21.15.88:0/1837154003 wait complete. 2024-09-21T02:03:54.992 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:03:54.992 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:03:55.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 -- 172.21.15.88:0/164640191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 msgr2=0x7ff8b00cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 --2- 172.21.15.88:0/164640191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 0x7ff8b00cacf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009830 tx=0x7ff8a402a510 comp rx=0 tx=0).stop 2024-09-21T02:03:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 -- 172.21.15.88:0/164640191 shutdown_connections 2024-09-21T02:03:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 --2- 172.21.15.88:0/164640191 >> v2:172.21.15.195:3300/0 conn(0x7ff8b00c49f0 0x7ff8b00c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 --2- 172.21.15.88:0/164640191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 0x7ff8b00cacf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 -- 172.21.15.88:0/164640191 >> 172.21.15.88:0/164640191 conn(0x7ff8b00690f0 msgr2=0x7ff8b0069500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 -- 172.21.15.88:0/164640191 shutdown_connections 2024-09-21T02:03:55.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.109+0000 7ff8b7371640 1 -- 172.21.15.88:0/164640191 wait complete. 2024-09-21T02:03:55.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.111+0000 7ff8b7371640 1 Processor -- start 2024-09-21T02:03:55.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.111+0000 7ff8b7371640 1 -- start start 2024-09-21T02:03:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b7371640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 0x7ff8b01e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b7371640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b00c49f0 0x7ff8b01e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b7371640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b0150220 0x7ff8b01e3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b7371640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0159b50 con 0x7ff8b00c49f0 2024-09-21T02:03:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b7371640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0159870 con 0x7ff8b0150220 2024-09-21T02:03:55.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b7371640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b00724a0 con 0x7ff8b00c4030 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b490c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b00c49f0 0x7ff8b01e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b490c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b00c49f0 0x7ff8b01e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60484/0 (socket says 172.21.15.88:60484) 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b490c640 1 -- 172.21.15.88:0/1137046340 learned_addr learned my addr 172.21.15.88:0/1137046340 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b590e640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b0150220 0x7ff8b01e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.112+0000 7ff8b510d640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 0x7ff8b01e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b490c640 1 -- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 msgr2=0x7ff8b01e3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b490c640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 0x7ff8b01e3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b490c640 1 -- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b0150220 msgr2=0x7ff8b01e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b490c640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b0150220 0x7ff8b01e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b490c640 1 -- 172.21.15.88:0/1137046340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b01e3fc0 con 0x7ff8b00c49f0 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b510d640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 0x7ff8b01e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b490c640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b00c49f0 0x7ff8b01e2bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8a402ab00 tx=0x7ff8a4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8adffb640 1 -- 172.21.15.88:0/1137046340 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4005c90 con 0x7ff8b00c49f0 2024-09-21T02:03:55.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8adffb640 1 -- 172.21.15.88:0/1137046340 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8a4031e50 con 0x7ff8b00c49f0 2024-09-21T02:03:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b01f05a0 con 0x7ff8b00c49f0 2024-09-21T02:03:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8adffb640 1 -- 172.21.15.88:0/1137046340 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4005260 con 0x7ff8b00c49f0 2024-09-21T02:03:55.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.113+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b01e5400 con 0x7ff8b00c49f0 2024-09-21T02:03:55.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.114+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b015a350 con 0x7ff8b00c49f0 2024-09-21T02:03:55.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.115+0000 7ff8adffb640 1 -- 172.21.15.88:0/1137046340 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff8a4005400 con 0x7ff8b00c49f0 2024-09-21T02:03:55.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.115+0000 7ff8adffb640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff89c078b90 0x7ff89c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:55.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.116+0000 7ff8adffb640 1 -- 172.21.15.88:0/1137046340 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff8a40bba60 con 0x7ff8b00c49f0 2024-09-21T02:03:55.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.116+0000 7ff8b510d640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff89c078b90 0x7ff89c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:55.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.116+0000 7ff8b510d640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff89c078b90 0x7ff89c07b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0007a50 tx=0x7ff8a0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:55.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.120+0000 7ff8adffb640 1 -- 172.21.15.88:0/1137046340 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff8a403f030 con 0x7ff8b00c49f0 2024-09-21T02:03:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:55.296+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff8b00c61f0 con 0x7ff8b00c49f0 2024-09-21T02:03:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.049+0000 7ff8adffb640 1 -- 172.21.15.88:0/1137046340 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7ff8a4083b20 con 0x7ff8b00c49f0 2024-09-21T02:03:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff89c078b90 msgr2=0x7ff89c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff89c078b90 0x7ff89c07b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0007a50 tx=0x7ff8a0009040 comp rx=0 tx=0).stop 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b00c49f0 msgr2=0x7ff8b01e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b00c49f0 0x7ff8b01e2bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8a402ab00 tx=0x7ff8a4033c20 comp rx=0 tx=0).stop 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 shutdown_connections 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff89c078b90 0x7ff89c07b0a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b0150220 0x7ff8b01e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b00c49f0 0x7ff8b01e2bc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 --2- 172.21.15.88:0/1137046340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b00c4030 0x7ff8b01e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.051+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 >> 172.21.15.88:0/1137046340 conn(0x7ff8b00690f0 msgr2=0x7ff8b0149cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.052+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 shutdown_connections 2024-09-21T02:03:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.052+0000 7ff8b7371640 1 -- 172.21.15.88:0/1137046340 wait complete. 2024-09-21T02:03:56.070 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:03:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.189+0000 7f8c04313640 1 -- 172.21.15.88:0/3210999885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc071f60 msgr2=0x7f8bfc15a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.189+0000 7f8c04313640 1 --2- 172.21.15.88:0/3210999885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc071f60 0x7f8bfc15a2e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bec005fd0 tx=0x7f8bec02ab20 comp rx=0 tx=0).stop 2024-09-21T02:03:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.189+0000 7f8c04313640 1 -- 172.21.15.88:0/3210999885 shutdown_connections 2024-09-21T02:03:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.189+0000 7f8c04313640 1 --2- 172.21.15.88:0/3210999885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc071f60 0x7f8bfc15a2e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.189+0000 7f8c04313640 1 -- 172.21.15.88:0/3210999885 >> 172.21.15.88:0/3210999885 conn(0x7f8bfc068d00 msgr2=0x7f8bfc069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.189+0000 7f8c04313640 1 -- 172.21.15.88:0/3210999885 shutdown_connections 2024-09-21T02:03:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.189+0000 7f8c04313640 1 -- 172.21.15.88:0/3210999885 wait complete. 2024-09-21T02:03:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.191+0000 7f8c04313640 1 Processor -- start 2024-09-21T02:03:56.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.191+0000 7f8c04313640 1 -- start start 2024-09-21T02:03:56.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c04313640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bfc0c5d90 0x7f8bfc0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c04313640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc0c8e60 0x7f8bfc1effd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c04313640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bfc14f750 0x7f8bfc146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c04313640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc153cb0 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c04313640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc159dc0 con 0x7f8bfc0c5d90 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c04313640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8bfc153e30 con 0x7f8bfc14f750 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c018ae640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc0c8e60 0x7f8bfc1effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c018ae640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc0c8e60 0x7f8bfc1effd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60504/0 (socket says 172.21.15.88:60504) 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c018ae640 1 -- 172.21.15.88:0/2924832141 learned_addr learned my addr 172.21.15.88:0/2924832141 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c020af640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bfc0c5d90 0x7f8bfc0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.192+0000 7f8c028b0640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bfc14f750 0x7f8bfc146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c018ae640 1 -- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bfc14f750 msgr2=0x7f8bfc146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c018ae640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bfc14f750 0x7f8bfc146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c018ae640 1 -- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bfc0c5d90 msgr2=0x7f8bfc0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c018ae640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bfc0c5d90 0x7f8bfc0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c018ae640 1 -- 172.21.15.88:0/2924832141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc1f34e0 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c018ae640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc0c8e60 0x7f8bfc1effd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c370 tx=0x7f8bf000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8bfaffd640 1 -- 172.21.15.88:0/2924832141 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0017070 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8bfaffd640 1 -- 172.21.15.88:0/2924832141 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bf00053c0 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8bfaffd640 1 -- 172.21.15.88:0/2924832141 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0003850 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc1f37c0 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.193+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc1f3d80 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.194+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bfc071f60 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.195+0000 7f8bfaffd640 1 -- 172.21.15.88:0/2924832141 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8bf001a070 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.195+0000 7f8bfaffd640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8be8078b90 0x7f8be807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.195+0000 7f8bfaffd640 1 -- 172.21.15.88:0/2924832141 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f8bf0099c80 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.195+0000 7f8c020af640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8be8078b90 0x7f8be807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.196+0000 7f8c020af640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8be8078b90 0x7f8be807b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bec035000 tx=0x7f8bec038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:56.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.200+0000 7f8bfaffd640 1 -- 172.21.15.88:0/2924832141 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8bf0061d70 con 0x7f8bfc0c8e60 2024-09-21T02:03:56.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:56.376+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8bfc0c9320 con 0x7f8bfc0c8e60 2024-09-21T02:03:57.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.055+0000 7f8bfaffd640 1 -- 172.21.15.88:0/2924832141 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f8bf00667a0 con 0x7f8bfc0c8e60 2024-09-21T02:03:57.057 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8be8078b90 msgr2=0x7f8be807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8be8078b90 0x7f8be807b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bec035000 tx=0x7f8bec038040 comp rx=0 tx=0).stop 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc0c8e60 msgr2=0x7f8bfc1effd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc0c8e60 0x7f8bfc1effd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c370 tx=0x7f8bf000c830 comp rx=0 tx=0).stop 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 shutdown_connections 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8be8078b90 0x7f8be807b0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bfc14f750 0x7f8bfc146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bfc0c8e60 0x7f8bfc1effd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 --2- 172.21.15.88:0/2924832141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bfc0c5d90 0x7f8bfc0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 >> 172.21.15.88:0/2924832141 conn(0x7f8bfc068d00 msgr2=0x7f8bfc1e2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:57.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 shutdown_connections 2024-09-21T02:03:57.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.058+0000 7f8c04313640 1 -- 172.21.15.88:0/2924832141 wait complete. 2024-09-21T02:03:57.082 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:03:57.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 -- 172.21.15.88:0/3837005734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 msgr2=0x7f8f6c0726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/3837005734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c0726d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0cfbd0 tx=0x7f8f6402ab40 comp rx=0 tx=0).stop 2024-09-21T02:03:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 -- 172.21.15.88:0/3837005734 shutdown_connections 2024-09-21T02:03:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/3837005734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c0726d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/3837005734 >> v2:172.21.15.195:3300/0 conn(0x7f8f6c071f60 0x7f8f6c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 -- 172.21.15.88:0/3837005734 >> 172.21.15.88:0/3837005734 conn(0x7f8f6c068d00 msgr2=0x7f8f6c069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 -- 172.21.15.88:0/3837005734 shutdown_connections 2024-09-21T02:03:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.198+0000 7f8f71c9b640 1 -- 172.21.15.88:0/3837005734 wait complete. 2024-09-21T02:03:57.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.200+0000 7f8f71c9b640 1 Processor -- start 2024-09-21T02:03:57.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.200+0000 7f8f71c9b640 1 -- start start 2024-09-21T02:03:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.201+0000 7f8f71c9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f6c0c5d90 0x7f8f6c151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.201+0000 7f8f71c9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.201+0000 7f8f71c9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f6c150780 0x7f8f6c154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.201+0000 7f8f71c9b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c153cb0 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.201+0000 7f8f71c9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c159dc0 con 0x7f8f6c150780 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.201+0000 7f8f71c9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c153e30 con 0x7f8f6c0c5d90 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60532/0 (socket says 172.21.15.88:60532) 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 -- 172.21.15.88:0/1222305028 learned_addr learned my addr 172.21.15.88:0/1222305028 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f6b7fe640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f6c0c5d90 0x7f8f6c151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f6bfff640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f6c150780 0x7f8f6c154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 -- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f6c0c5d90 msgr2=0x7f8f6c151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f6c0c5d90 0x7f8f6c151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 -- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f6c150780 msgr2=0x7f8f6c154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f6c150780 0x7f8f6c154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 -- 172.21.15.88:0/1222305028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c151210 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f62ffd640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c157d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c370 tx=0x7f8f5400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f68ff9640 1 -- 172.21.15.88:0/1222305028 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54017070 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f68ff9640 1 -- 172.21.15.88:0/1222305028 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f540053c0 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f68ff9640 1 -- 172.21.15.88:0/1222305028 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54003850 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c1514f0 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.202+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1f0320 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.203+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c153970 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.204+0000 7f8f68ff9640 1 -- 172.21.15.88:0/1222305028 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f5401a070 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.205+0000 7f8f68ff9640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8f4c078bc0 0x7f8f4c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.205+0000 7f8f68ff9640 1 -- 172.21.15.88:0/1222305028 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f8f54099ec0 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.205+0000 7f8f6b7fe640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8f4c078bc0 0x7f8f4c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.205+0000 7f8f6b7fe640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8f4c078bc0 0x7f8f4c07b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c004630 tx=0x7f8f5c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:57.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.209+0000 7f8f68ff9640 1 -- 172.21.15.88:0/1222305028 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f6c153970 con 0x7f8f6c0c8e60 2024-09-21T02:03:57.329 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:03:57.332 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:03:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:57.428+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8f6c071f60 con 0x7f8f6c0c8e60 2024-09-21T02:03:58.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.058+0000 7f8f68ff9640 1 -- 172.21.15.88:0/1222305028 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f8f54061db0 con 0x7f8f6c0c8e60 2024-09-21T02:03:58.058 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T02:03:58.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.062+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8f4c078bc0 msgr2=0x7f8f4c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.062+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8f4c078bc0 0x7f8f4c07b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c004630 tx=0x7f8f5c00b2e0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.062+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 msgr2=0x7f8f6c157d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.062+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c157d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c370 tx=0x7f8f5400c830 comp rx=0 tx=0).stop 2024-09-21T02:03:58.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.063+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 shutdown_connections 2024-09-21T02:03:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.063+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8f4c078bc0 0x7f8f4c07b0d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.063+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f6c150780 0x7f8f6c154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.063+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f6c0c8e60 0x7f8f6c157d00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.063+0000 7f8f71c9b640 1 --2- 172.21.15.88:0/1222305028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f6c0c5d90 0x7f8f6c151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.063+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 >> 172.21.15.88:0/1222305028 conn(0x7f8f6c068d00 msgr2=0x7f8f6c0c7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.063+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 shutdown_connections 2024-09-21T02:03:58.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.064+0000 7f8f71c9b640 1 -- 172.21.15.88:0/1222305028 wait complete. 2024-09-21T02:03:58.088 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:03:58.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.202+0000 7fb8518b4640 1 -- 172.21.15.88:0/4122309836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c0cbbd0 msgr2=0x7fb84c0cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.202+0000 7fb8518b4640 1 --2- 172.21.15.88:0/4122309836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c0cbbd0 0x7fb84c0cbfe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb844009be0 tx=0x7fb84402a960 comp rx=0 tx=0).stop 2024-09-21T02:03:58.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.202+0000 7fb8518b4640 1 -- 172.21.15.88:0/4122309836 shutdown_connections 2024-09-21T02:03:58.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.202+0000 7fb8518b4640 1 --2- 172.21.15.88:0/4122309836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c0cbbd0 0x7fb84c0cbfe0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.202+0000 7fb8518b4640 1 -- 172.21.15.88:0/4122309836 >> 172.21.15.88:0/4122309836 conn(0x7fb84c0cd9f0 msgr2=0x7fb84c0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.202+0000 7fb8518b4640 1 -- 172.21.15.88:0/4122309836 shutdown_connections 2024-09-21T02:03:58.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.202+0000 7fb8518b4640 1 -- 172.21.15.88:0/4122309836 wait complete. 2024-09-21T02:03:58.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.203+0000 7fb8518b4640 1 Processor -- start 2024-09-21T02:03:58.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.204+0000 7fb8518b4640 1 -- start start 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb8518b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb84c0cbbd0 0x7fb84c1e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb8518b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb84c0ccd00 0x7fb84c1e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb8518b4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c15f970 0x7fb84c1532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb8518b4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c0cc520 con 0x7fb84c15f970 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb8518b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c15c640 con 0x7fb84c0ccd00 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb8518b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c15fd80 con 0x7fb84c0cbbd0 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c15f970 0x7fb84c1532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c15f970 0x7fb84c1532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52408/0 (socket says 172.21.15.88:52408) 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 -- 172.21.15.88:0/3187842314 learned_addr learned my addr 172.21.15.88:0/3187842314 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84affd640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb84c0cbbd0 0x7fb84c1e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84a7fc640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb84c0ccd00 0x7fb84c1e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 -- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb84c0cbbd0 msgr2=0x7fb84c1e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb84c0cbbd0 0x7fb84c1e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 -- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb84c0ccd00 msgr2=0x7fb84c1e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb84c0ccd00 0x7fb84c1e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 -- 172.21.15.88:0/3187842314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c1e5330 con 0x7fb84c15f970 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.205+0000 7fb84b7fe640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c15f970 0x7fb84c1532a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb844005fd0 tx=0x7fb844003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.206+0000 7fb833fff640 1 -- 172.21.15.88:0/3187842314 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844030e60 con 0x7fb84c15f970 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.206+0000 7fb833fff640 1 -- 172.21.15.88:0/3187842314 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb84403e070 con 0x7fb84c15f970 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.206+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c0c5730 con 0x7fb84c15f970 2024-09-21T02:03:58.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.206+0000 7fb833fff640 1 -- 172.21.15.88:0/3187842314 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844032310 con 0x7fb84c15f970 2024-09-21T02:03:58.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.206+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c0c5cf0 con 0x7fb84c15f970 2024-09-21T02:03:58.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.207+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84c0c5480 con 0x7fb84c15f970 2024-09-21T02:03:58.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.207+0000 7fb833fff640 1 -- 172.21.15.88:0/3187842314 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8440324b0 con 0x7fb84c15f970 2024-09-21T02:03:58.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.208+0000 7fb833fff640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb838078bc0 0x7fb83807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.208+0000 7fb833fff640 1 -- 172.21.15.88:0/3187842314 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8440bc730 con 0x7fb84c15f970 2024-09-21T02:03:58.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.208+0000 7fb84affd640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb838078bc0 0x7fb83807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.209+0000 7fb84affd640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb838078bc0 0x7fb83807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb83c007fd0 tx=0x7fb83c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.212+0000 7fb833fff640 1 -- 172.21.15.88:0/3187842314 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb84402d070 con 0x7fb84c15f970 2024-09-21T02:03:58.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.392+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb84c06d550 con 0x7fb84c15f970 2024-09-21T02:03:58.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.393+0000 7fb833fff640 1 -- 172.21.15.88:0/3187842314 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16678 (secure 0 0 0) 0x7fb844084b70 con 0x7fb84c15f970 2024-09-21T02:03:58.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:03:58.394 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:57.430130+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.395+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb838078bc0 msgr2=0x7fb83807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.395+0000 7fb8518b4640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb838078bc0 0x7fb83807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb83c007fd0 tx=0x7fb83c009040 comp rx=0 tx=0).stop 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.395+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c15f970 msgr2=0x7fb84c1532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.395+0000 7fb8518b4640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c15f970 0x7fb84c1532a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb844005fd0 tx=0x7fb844003c90 comp rx=0 tx=0).stop 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 shutdown_connections 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb838078bc0 0x7fb83807b0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84c15f970 0x7fb84c1532a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb84c0ccd00 0x7fb84c1e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 --2- 172.21.15.88:0/3187842314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb84c0cbbd0 0x7fb84c1e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 >> 172.21.15.88:0/3187842314 conn(0x7fb84c0cd9f0 msgr2=0x7fb84c1554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 shutdown_connections 2024-09-21T02:03:58.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.396+0000 7fb8518b4640 1 -- 172.21.15.88:0/3187842314 wait complete. 2024-09-21T02:03:58.407 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.517+0000 7fae9e6ec640 1 -- 172.21.15.88:0/870859473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 msgr2=0x7fae980cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.517+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/870859473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980cbbd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae90005fd0 tx=0x7fae9002ab20 comp rx=0 tx=0).stop 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.517+0000 7fae9e6ec640 1 -- 172.21.15.88:0/870859473 shutdown_connections 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.517+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/870859473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980cbbd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.517+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/870859473 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae980ccd00 0x7fae9815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.517+0000 7fae9e6ec640 1 -- 172.21.15.88:0/870859473 >> 172.21.15.88:0/870859473 conn(0x7fae980cd9f0 msgr2=0x7fae980cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.518+0000 7fae9e6ec640 1 -- 172.21.15.88:0/870859473 shutdown_connections 2024-09-21T02:03:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.518+0000 7fae9e6ec640 1 -- 172.21.15.88:0/870859473 wait complete. 2024-09-21T02:03:58.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.519+0000 7fae9e6ec640 1 Processor -- start 2024-09-21T02:03:58.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.519+0000 7fae9e6ec640 1 -- start start 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae980cad60 0x7fae981e5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae980ccd00 0x7fae9815e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae9806d470 con 0x7fae980cb170 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae9815c640 con 0x7fae980ccd00 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fae9806d5f0 con 0x7fae980cad60 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52424/0 (socket says 172.21.15.88:52424) 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 -- 172.21.15.88:0/2419588342 learned_addr learned my addr 172.21.15.88:0/2419588342 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae97fff640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae980cad60 0x7fae981e5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9cc89640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae980ccd00 0x7fae9815e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 -- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae980cad60 msgr2=0x7fae981e5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae980cad60 0x7fae981e5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 -- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae980ccd00 msgr2=0x7fae9815e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae980ccd00 0x7fae9815e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 -- 172.21.15.88:0/2419588342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae981e6120 con 0x7fae980cb170 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae977fe640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980d00b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00c3f0 tx=0x7fae8c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae94ff9640 1 -- 172.21.15.88:0/2419588342 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c017070 con 0x7fae980cb170 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae94ff9640 1 -- 172.21.15.88:0/2419588342 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae8c0053e0 con 0x7fae980cb170 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae981e6960 con 0x7fae980cb170 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae94ff9640 1 -- 172.21.15.88:0/2419588342 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c003870 con 0x7fae980cb170 2024-09-21T02:03:58.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.521+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981e53e0 con 0x7fae980cb170 2024-09-21T02:03:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.522+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9806de40 con 0x7fae980cb170 2024-09-21T02:03:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.523+0000 7fae94ff9640 1 -- 172.21.15.88:0/2419588342 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae8c01a070 con 0x7fae980cb170 2024-09-21T02:03:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.523+0000 7fae94ff9640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fae84078b90 0x7fae8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.524+0000 7fae94ff9640 1 -- 172.21.15.88:0/2419588342 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae8c099f20 con 0x7fae980cb170 2024-09-21T02:03:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.524+0000 7fae97fff640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fae84078b90 0x7fae8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.524+0000 7fae97fff640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fae84078b90 0x7fae8407b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae8800bb50 tx=0x7fae8800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.528+0000 7fae94ff9640 1 -- 172.21.15.88:0/2419588342 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae8c061e30 con 0x7fae980cb170 2024-09-21T02:03:58.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.703+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae98155f30 con 0x7fae980cb170 2024-09-21T02:03:58.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.704+0000 7fae94ff9640 1 -- 172.21.15.88:0/2419588342 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16678 (secure 0 0 0) 0x7fae8c066860 con 0x7fae980cb170 2024-09-21T02:03:58.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:03:58.704 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:57.430130+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:03:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fae84078b90 msgr2=0x7fae8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fae84078b90 0x7fae8407b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae8800bb50 tx=0x7fae8800b2e0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 msgr2=0x7fae980d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980d00b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00c3f0 tx=0x7fae8c00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 shutdown_connections 2024-09-21T02:03:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fae84078b90 0x7fae8407b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae980ccd00 0x7fae9815e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae980cb170 0x7fae980d00b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 --2- 172.21.15.88:0/2419588342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae980cad60 0x7fae981e5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 >> 172.21.15.88:0/2419588342 conn(0x7fae980cd9f0 msgr2=0x7fae980cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 shutdown_connections 2024-09-21T02:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.706+0000 7fae9e6ec640 1 -- 172.21.15.88:0/2419588342 wait complete. 2024-09-21T02:03:58.717 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:03:58.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.832+0000 7f881b8f9640 1 -- 172.21.15.88:0/160028437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8800005350 msgr2=0x7f880000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.832+0000 7f881b8f9640 1 --2- 172.21.15.88:0/160028437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8800005350 0x7f880000b9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88140b3630 tx=0x7f8800005830 comp rx=0 tx=0).stop 2024-09-21T02:03:58.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.832+0000 7f881b8f9640 1 -- 172.21.15.88:0/160028437 shutdown_connections 2024-09-21T02:03:58.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.832+0000 7f881b8f9640 1 --2- 172.21.15.88:0/160028437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8800005350 0x7f880000b9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.832+0000 7f881b8f9640 1 -- 172.21.15.88:0/160028437 >> 172.21.15.88:0/160028437 conn(0x7f881406de70 msgr2=0x7f881406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:58.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.832+0000 7f881b8f9640 1 -- 172.21.15.88:0/160028437 shutdown_connections 2024-09-21T02:03:58.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.832+0000 7f881b8f9640 1 -- 172.21.15.88:0/160028437 wait complete. 2024-09-21T02:03:58.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.834+0000 7f881b8f9640 1 Processor -- start 2024-09-21T02:03:58.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.834+0000 7f881b8f9640 1 -- start start 2024-09-21T02:03:58.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f881b8f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8800005350 0x7f881414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f881b8f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8814148240 0x7f8814148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f881b8f9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8814155590 0x7f88141559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f881b8f9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f88140c9c50 con 0x7f8814155590 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f881b8f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88140c9970 con 0x7f8814148240 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f881b8f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8814072190 con 0x7f8800005350 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f8819e96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8814155590 0x7f88141559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f8819e96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8814155590 0x7f88141559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52442/0 (socket says 172.21.15.88:52442) 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.835+0000 7f8819e96640 1 -- 172.21.15.88:0/2624961379 learned_addr learned my addr 172.21.15.88:0/2624961379 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8818e94640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8814148240 0x7f8814148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819695640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8800005350 0x7f881414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819e96640 1 -- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8800005350 msgr2=0x7f881414a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819e96640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8800005350 0x7f881414a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819e96640 1 -- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8814148240 msgr2=0x7f8814148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819e96640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8814148240 0x7f8814148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819e96640 1 -- 172.21.15.88:0/2624961379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881414acc0 con 0x7f8814155590 2024-09-21T02:03:58.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819695640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8800005350 0x7f881414a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:03:58.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8819e96640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8814155590 0x7f88141559a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f880c009ee0 tx=0x7f880c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8809ffb640 1 -- 172.21.15.88:0/2624961379 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c038040 con 0x7f8814155590 2024-09-21T02:03:58.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8809ffb640 1 -- 172.21.15.88:0/2624961379 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f880c02b630 con 0x7f8814155590 2024-09-21T02:03:58.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141f2bd0 con 0x7f8814155590 2024-09-21T02:03:58.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f8809ffb640 1 -- 172.21.15.88:0/2624961379 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c004d60 con 0x7f8814155590 2024-09-21T02:03:58.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.836+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881414b970 con 0x7f8814155590 2024-09-21T02:03:58.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.837+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88140c3620 con 0x7f8814155590 2024-09-21T02:03:58.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.838+0000 7f8809ffb640 1 -- 172.21.15.88:0/2624961379 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f880c043020 con 0x7f8814155590 2024-09-21T02:03:58.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.838+0000 7f8809ffb640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f87f8078b90 0x7f87f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:58.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.838+0000 7f8809ffb640 1 -- 172.21.15.88:0/2624961379 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f880c0bac90 con 0x7f8814155590 2024-09-21T02:03:58.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.839+0000 7f8819695640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f87f8078b90 0x7f87f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:58.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.839+0000 7f8819695640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f87f8078b90 0x7f87f807b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8810000c00 tx=0x7f8810006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:58.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:58.843+0000 7f8809ffb640 1 -- 172.21.15.88:0/2624961379 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f880c0831f0 con 0x7f8814155590 2024-09-21T02:03:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.057+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f88140c3370 con 0x7f8814155590 2024-09-21T02:03:59.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.070+0000 7f8809ffb640 1 -- 172.21.15.88:0/2624961379 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f880c082af0 con 0x7f8814155590 2024-09-21T02:03:59.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f87f8078b90 msgr2=0x7f87f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f87f8078b90 0x7f87f807b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8810000c00 tx=0x7f8810006b70 comp rx=0 tx=0).stop 2024-09-21T02:03:59.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8814155590 msgr2=0x7f88141559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8814155590 0x7f88141559a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f880c009ee0 tx=0x7f880c033670 comp rx=0 tx=0).stop 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 shutdown_connections 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f87f8078b90 0x7f87f807b0a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8814155590 0x7f88141559a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8814148240 0x7f8814148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 --2- 172.21.15.88:0/2624961379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8800005350 0x7f881414a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 >> 172.21.15.88:0/2624961379 conn(0x7f881406de70 msgr2=0x7f8814158330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 shutdown_connections 2024-09-21T02:03:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.073+0000 7f881b8f9640 1 -- 172.21.15.88:0/2624961379 wait complete. 2024-09-21T02:03:59.084 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300} 2024-09-21T02:03:59.085 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.198+0000 7f8786320640 1 -- 172.21.15.88:0/3764533044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 msgr2=0x7f87800cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.198+0000 7f8786320640 1 --2- 172.21.15.88:0/3764533044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87800cc050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8778005fd0 tx=0x7f877802aa90 comp rx=0 tx=0).stop 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.199+0000 7f8786320640 1 -- 172.21.15.88:0/3764533044 shutdown_connections 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.199+0000 7f8786320640 1 --2- 172.21.15.88:0/3764533044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87800cc050 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.199+0000 7f8786320640 1 --2- 172.21.15.88:0/3764533044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f878015f070 0x7f8780074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.199+0000 7f8786320640 1 -- 172.21.15.88:0/3764533044 >> 172.21.15.88:0/3764533044 conn(0x7f87800d16b0 msgr2=0x7f87800cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.199+0000 7f8786320640 1 -- 172.21.15.88:0/3764533044 shutdown_connections 2024-09-21T02:03:59.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.199+0000 7f8786320640 1 -- 172.21.15.88:0/3764533044 wait complete. 2024-09-21T02:03:59.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.200+0000 7f8786320640 1 Processor -- start 2024-09-21T02:03:59.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.200+0000 7f8786320640 1 -- start start 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87801575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87800cd140 0x7f87801f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878015f070 0x7f87801554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f878015bb00 con 0x7f87800749d0 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f87800cbb30 con 0x7f878015f070 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f878015bc80 con 0x7f87800cd140 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87801575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87801575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52472/0 (socket says 172.21.15.88:52472) 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 -- 172.21.15.88:0/2773237550 learned_addr learned my addr 172.21.15.88:0/2773237550 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 -- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87800cd140 msgr2=0x7f87801f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87800cd140 0x7f87801f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 -- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878015f070 msgr2=0x7f87801554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878015f070 0x7f87801554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 -- 172.21.15.88:0/2773237550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87801f5110 con 0x7f87800749d0 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877ffff640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87801575d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f877000c3f0 tx=0x7f877000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877cff9640 1 -- 172.21.15.88:0/2773237550 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770017070 con 0x7f87800749d0 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87801f16f0 con 0x7f87800749d0 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877cff9640 1 -- 172.21.15.88:0/2773237550 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87700035f0 con 0x7f87800749d0 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f877cff9640 1 -- 172.21.15.88:0/2773237550 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770003a40 con 0x7f87800749d0 2024-09-21T02:03:59.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.202+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87801f1cb0 con 0x7f87800749d0 2024-09-21T02:03:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.203+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87800ce880 con 0x7f87800749d0 2024-09-21T02:03:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.204+0000 7f877cff9640 1 -- 172.21.15.88:0/2773237550 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f877001a070 con 0x7f87800749d0 2024-09-21T02:03:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.204+0000 7f877cff9640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f876c078b90 0x7f876c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:03:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.205+0000 7f877cff9640 1 -- 172.21.15.88:0/2773237550 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f877009a130 con 0x7f87800749d0 2024-09-21T02:03:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.205+0000 7f877f7fe640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f876c078b90 0x7f876c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:03:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.205+0000 7f877f7fe640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f876c078b90 0x7f876c07b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87800c4940 tx=0x7f8774005bd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:03:59.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.209+0000 7f877cff9640 1 -- 172.21.15.88:0/2773237550 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87700624f0 con 0x7f87800749d0 2024-09-21T02:03:59.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:03:59.424+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f87800cefd0 con 0x7f87800749d0 2024-09-21T02:04:00.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.072+0000 7f877cff9640 1 -- 172.21.15.88:0/2773237550 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f8770061df0 con 0x7f87800749d0 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.074+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f876c078b90 msgr2=0x7f876c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.074+0000 7f8786320640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f876c078b90 0x7f876c07b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87800c4940 tx=0x7f8774005bd0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.074+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 msgr2=0x7f87801575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.074+0000 7f8786320640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87801575d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f877000c3f0 tx=0x7f877000c8b0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.074+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 shutdown_connections 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.075+0000 7f8786320640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f876c078b90 0x7f876c07b0a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.075+0000 7f8786320640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878015f070 0x7f87801554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.075+0000 7f8786320640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87800cd140 0x7f87801f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.075+0000 7f8786320640 1 --2- 172.21.15.88:0/2773237550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87800749d0 0x7f87801575d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.075+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 >> 172.21.15.88:0/2773237550 conn(0x7f87800d16b0 msgr2=0x7f87800cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.075+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 shutdown_connections 2024-09-21T02:04:00.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.075+0000 7f8786320640 1 -- 172.21.15.88:0/2773237550 wait complete. 2024-09-21T02:04:00.089 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:04:00.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.208+0000 7f080a48b640 1 -- 172.21.15.88:0/292251414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07f0005300 msgr2=0x7f07f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.208+0000 7f080a48b640 1 --2- 172.21.15.88:0/292251414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07f0005300 0x7f07f000b9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07fc005fd0 tx=0x7f07f0005830 comp rx=0 tx=0).stop 2024-09-21T02:04:00.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.208+0000 7f080a48b640 1 -- 172.21.15.88:0/292251414 shutdown_connections 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.208+0000 7f080a48b640 1 --2- 172.21.15.88:0/292251414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07f0005300 0x7f07f000b9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.208+0000 7f080a48b640 1 -- 172.21.15.88:0/292251414 >> 172.21.15.88:0/292251414 conn(0x7f08040720b0 msgr2=0x7f08040cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.208+0000 7f080a48b640 1 -- 172.21.15.88:0/292251414 shutdown_connections 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.208+0000 7f080a48b640 1 -- 172.21.15.88:0/292251414 wait complete. 2024-09-21T02:04:00.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.210+0000 7f080a48b640 1 Processor -- start 2024-09-21T02:04:00.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.210+0000 7f080a48b640 1 -- start start 2024-09-21T02:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f080a48b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f0005300 0x7f080415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f080a48b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080414f2e0 0x7f0804153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f080a48b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0804159eb0 0x7f080415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f080a48b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0804159190 con 0x7f080414f2e0 2024-09-21T02:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f080a48b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08040c4930 con 0x7f0804159eb0 2024-09-21T02:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f080a48b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f080406f1f0 con 0x7f07f0005300 2024-09-21T02:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080414f2e0 0x7f0804153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080414f2e0 0x7f0804153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52498/0 (socket says 172.21.15.88:52498) 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 -- 172.21.15.88:0/700537951 learned_addr learned my addr 172.21.15.88:0/700537951 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f0808a28640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0804159eb0 0x7f080415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f0803fff640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f0005300 0x7f080415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 -- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f0005300 msgr2=0x7f080415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f0005300 0x7f080415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 -- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0804159eb0 msgr2=0x7f080415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0804159eb0 0x7f080415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f07fbfff640 1 -- 172.21.15.88:0/700537951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080415c000 con 0x7f080414f2e0 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f0808a28640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0804159eb0 0x7f080415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.211+0000 7f0803fff640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f0005300 0x7f080415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.212+0000 7f07fbfff640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080414f2e0 0x7f0804153ee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07ec003ed0 tx=0x7f07ec001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.212+0000 7f08017fa640 1 -- 172.21.15.88:0/700537951 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec015070 con 0x7f080414f2e0 2024-09-21T02:04:00.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.212+0000 7f08017fa640 1 -- 172.21.15.88:0/700537951 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07ec011840 con 0x7f080414f2e0 2024-09-21T02:04:00.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.212+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08041f36d0 con 0x7f080414f2e0 2024-09-21T02:04:00.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.212+0000 7f08017fa640 1 -- 172.21.15.88:0/700537951 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec011c90 con 0x7f080414f2e0 2024-09-21T02:04:00.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.212+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08040ca2a0 con 0x7f080414f2e0 2024-09-21T02:04:00.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.213+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0804152a70 con 0x7f080414f2e0 2024-09-21T02:04:00.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.213+0000 7f08017fa640 1 -- 172.21.15.88:0/700537951 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07ec018070 con 0x7f080414f2e0 2024-09-21T02:04:00.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.214+0000 7f08017fa640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f07e4078bc0 0x7f07e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:00.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.214+0000 7f08017fa640 1 -- 172.21.15.88:0/700537951 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f07ec09b9b0 con 0x7f080414f2e0 2024-09-21T02:04:00.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.214+0000 7f0803fff640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f07e4078bc0 0x7f07e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:00.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.215+0000 7f0803fff640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f07e4078bc0 0x7f07e407b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07f4001010 tx=0x7f07f4006950 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:00.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.218+0000 7f08017fa640 1 -- 172.21.15.88:0/700537951 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f07ec063de0 con 0x7f080414f2e0 2024-09-21T02:04:00.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:00.432+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f080406d810 con 0x7f080414f2e0 2024-09-21T02:04:01.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.081+0000 7f08017fa640 1 -- 172.21.15.88:0/700537951 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f07ec0636e0 con 0x7f080414f2e0 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f07e4078bc0 msgr2=0x7f07e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f07e4078bc0 0x7f07e407b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07f4001010 tx=0x7f07f4006950 comp rx=0 tx=0).stop 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080414f2e0 msgr2=0x7f0804153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080414f2e0 0x7f0804153ee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07ec003ed0 tx=0x7f07ec001630 comp rx=0 tx=0).stop 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 shutdown_connections 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f07e4078bc0 0x7f07e407b0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0804159eb0 0x7f080415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080414f2e0 0x7f0804153ee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 --2- 172.21.15.88:0/700537951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f0005300 0x7f080415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 >> 172.21.15.88:0/700537951 conn(0x7f08040720b0 msgr2=0x7f080406cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 shutdown_connections 2024-09-21T02:04:01.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.084+0000 7f080a48b640 1 -- 172.21.15.88:0/700537951 wait complete. 2024-09-21T02:04:01.096 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.213+0000 7f8adeb8b640 1 -- 172.21.15.88:0/1124642454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ac0003a00 msgr2=0x7f8ac0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.213+0000 7f8adeb8b640 1 --2- 172.21.15.88:0/1124642454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ac0003a00 0x7f8ac0003e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009b50 tx=0x7f8ac0005830 comp rx=0 tx=0).stop 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.213+0000 7f8add128640 1 -- 172.21.15.88:0/1124642454 reap_dead start 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.214+0000 7f8adeb8b640 1 -- 172.21.15.88:0/1124642454 shutdown_connections 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.214+0000 7f8adeb8b640 1 -- 172.21.15.88:0/1124642454 >> 172.21.15.88:0/1124642454 conn(0x7f8ad80cda90 msgr2=0x7f8ad80cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:01.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.214+0000 7f8adeb8b640 1 -- 172.21.15.88:0/1124642454 shutdown_connections 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.214+0000 7f8adeb8b640 1 -- 172.21.15.88:0/1124642454 wait complete. 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.215+0000 7f8adeb8b640 1 Processor -- start 2024-09-21T02:04:01.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.215+0000 7f8adeb8b640 1 -- start start 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ac0003a00 0x7f8ad81f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad81532b0 0x7f8ad81529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad8156d70 0x7f8ad8157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80b46f0 con 0x7f8ad81532b0 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80b4410 con 0x7f8ac0003a00 2024-09-21T02:04:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad806d470 con 0x7f8ad8156d70 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad81532b0 0x7f8ad81529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad81532b0 0x7f8ad81529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52514/0 (socket says 172.21.15.88:52514) 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 -- 172.21.15.88:0/2642281767 learned_addr learned my addr 172.21.15.88:0/2642281767 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adc927640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ac0003a00 0x7f8ad81f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8add128640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad8156d70 0x7f8ad8157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 -- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad8156d70 msgr2=0x7f8ad8157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad8156d70 0x7f8ad8157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 -- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ac0003a00 msgr2=0x7f8ad81f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ac0003a00 0x7f8ad81f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 -- 172.21.15.88:0/2642281767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad81f2c10 con 0x7f8ad81532b0 2024-09-21T02:04:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad7fff640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad81532b0 0x7f8ad81529f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8acc011a50 tx=0x7f8acc011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad57fa640 1 -- 172.21.15.88:0/2642281767 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc01f070 con 0x7f8ad81532b0 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad57fa640 1 -- 172.21.15.88:0/2642281767 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8acc00fe80 con 0x7f8ad81532b0 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad81f21d0 con 0x7f8ad81532b0 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8ad57fa640 1 -- 172.21.15.88:0/2642281767 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc012690 con 0x7f8ad81532b0 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.217+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad80c73f0 con 0x7f8ad81532b0 2024-09-21T02:04:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.218+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad80b4c10 con 0x7f8ad81532b0 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.219+0000 7f8ad57fa640 1 -- 172.21.15.88:0/2642281767 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8acc022070 con 0x7f8ad81532b0 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.219+0000 7f8ad57fa640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8ac4078bc0 0x7f8ac407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.220+0000 7f8ad57fa640 1 -- 172.21.15.88:0/2642281767 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8acc09b780 con 0x7f8ad81532b0 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.220+0000 7f8adc927640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8ac4078bc0 0x7f8ac407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.220+0000 7f8adc927640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8ac4078bc0 0x7f8ac407b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8035000 tx=0x7f8ac802d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:01.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.224+0000 7f8ad57fa640 1 -- 172.21.15.88:0/2642281767 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8acc063610 con 0x7f8ad81532b0 2024-09-21T02:04:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:01.439+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8ad80c4920 con 0x7f8ad81532b0 2024-09-21T02:04:02.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.083+0000 7f8ad57fa640 1 -- 172.21.15.88:0/2642281767 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f8acc068040 con 0x7f8ad81532b0 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8ac4078bc0 msgr2=0x7f8ac407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8ac4078bc0 0x7f8ac407b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8035000 tx=0x7f8ac802d040 comp rx=0 tx=0).stop 2024-09-21T02:04:02.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad81532b0 msgr2=0x7f8ad81529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad81532b0 0x7f8ad81529f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8acc011a50 tx=0x7f8acc011f10 comp rx=0 tx=0).stop 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 shutdown_connections 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8ac4078bc0 0x7f8ac407b0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad8156d70 0x7f8ad8157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad81532b0 0x7f8ad81529f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 --2- 172.21.15.88:0/2642281767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ac0003a00 0x7f8ad81f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 >> 172.21.15.88:0/2642281767 conn(0x7f8ad80cda90 msgr2=0x7f8ad80cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 shutdown_connections 2024-09-21T02:04:02.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.085+0000 7f8adeb8b640 1 -- 172.21.15.88:0/2642281767 wait complete. 2024-09-21T02:04:02.097 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:04:02.097 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.209+0000 7f76f89df640 1 -- 172.21.15.88:0/1580460000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e0004800 msgr2=0x7f76e0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.209+0000 7f76f89df640 1 --2- 172.21.15.88:0/1580460000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e0004800 0x7f76e0004c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f76f40733f0 tx=0x7f76e0005830 comp rx=0 tx=0).stop 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.209+0000 7f76f89df640 1 -- 172.21.15.88:0/1580460000 shutdown_connections 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.209+0000 7f76f89df640 1 --2- 172.21.15.88:0/1580460000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76e0004800 0x7f76e0004c10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.209+0000 7f76f89df640 1 -- 172.21.15.88:0/1580460000 >> 172.21.15.88:0/1580460000 conn(0x7f76f40c9f90 msgr2=0x7f76f40ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.209+0000 7f76f89df640 1 -- 172.21.15.88:0/1580460000 shutdown_connections 2024-09-21T02:04:02.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.209+0000 7f76f89df640 1 -- 172.21.15.88:0/1580460000 wait complete. 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.211+0000 7f76f89df640 1 Processor -- start 2024-09-21T02:04:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.211+0000 7f76f89df640 1 -- start start 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f89df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e0004800 0x7f76f4150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f89df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f4151710 0x7f76f4149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f89df640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f414a3e0 0x7f76f415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f89df640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f76f40c72f0 con 0x7f76f414a3e0 2024-09-21T02:04:02.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f89df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f76f40c4e10 con 0x7f76e0004800 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f89df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f76f4068d00 con 0x7f76f4151710 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f414a3e0 0x7f76f415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f414a3e0 0x7f76f415a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52534/0 (socket says 172.21.15.88:52534) 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f2d9d640 1 -- 172.21.15.88:0/11062921 learned_addr learned my addr 172.21.15.88:0/11062921 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76f259c640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e0004800 0x7f76f4150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.212+0000 7f76e9d9b640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f4151710 0x7f76f4149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f2d9d640 1 -- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f4151710 msgr2=0x7f76f4149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f2d9d640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f4151710 0x7f76f4149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f2d9d640 1 -- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e0004800 msgr2=0x7f76f4150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f2d9d640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e0004800 0x7f76f4150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f2d9d640 1 -- 172.21.15.88:0/11062921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f414a7f0 con 0x7f76f414a3e0 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f259c640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e0004800 0x7f76f4150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f2d9d640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f414a3e0 0x7f76f415a900 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76ec005fd0 tx=0x7f76ec010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76eb7fe640 1 -- 172.21.15.88:0/11062921 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004b20 con 0x7f76f414a3e0 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76eb7fe640 1 -- 172.21.15.88:0/11062921 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76ec015030 con 0x7f76f414a3e0 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41511c0 con 0x7f76f414a3e0 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76eb7fe640 1 -- 172.21.15.88:0/11062921 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec003970 con 0x7f76f414a3e0 2024-09-21T02:04:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.213+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41514a0 con 0x7f76f414a3e0 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.214+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f415bb00 con 0x7f76f414a3e0 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.215+0000 7f76eb7fe640 1 -- 172.21.15.88:0/11062921 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76ec003b10 con 0x7f76f414a3e0 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.215+0000 7f76eb7fe640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f76d4078b90 0x7f76d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.215+0000 7f76eb7fe640 1 -- 172.21.15.88:0/11062921 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f76ec09bb00 con 0x7f76f414a3e0 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.215+0000 7f76f259c640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f76d4078b90 0x7f76d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.216+0000 7f76f259c640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f76d4078b90 0x7f76d407b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76e4006990 tx=0x7f76e4006900 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.220+0000 7f76eb7fe640 1 -- 172.21.15.88:0/11062921 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76ec063f40 con 0x7f76f414a3e0 2024-09-21T02:04:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.432+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f41582f0 con 0x7f76f414a3e0 2024-09-21T02:04:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.433+0000 7f76eb7fe640 1 -- 172.21.15.88:0/11062921 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6861 (secure 0 0 0) 0x7f76ec063840 con 0x7f76f414a3e0 2024-09-21T02:04:02.433 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:02.433 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T02:04:02:081550+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4398,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:01.439919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0],"up":{"mds_0":4401},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T02:04:02.434 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T02:04:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f76d4078b90 msgr2=0x7f76d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f76d4078b90 0x7f76d407b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76e4006990 tx=0x7f76e4006900 comp rx=0 tx=0).stop 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f414a3e0 msgr2=0x7f76f415a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f414a3e0 0x7f76f415a900 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76ec005fd0 tx=0x7f76ec010f40 comp rx=0 tx=0).stop 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 shutdown_connections 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f76d4078b90 0x7f76d407b0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76f414a3e0 0x7f76f415a900 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76f4151710 0x7f76f4149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 --2- 172.21.15.88:0/11062921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e0004800 0x7f76f4150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 >> 172.21.15.88:0/11062921 conn(0x7f76f40c9f90 msgr2=0x7f76f40c84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 shutdown_connections 2024-09-21T02:04:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.435+0000 7f76f89df640 1 -- 172.21.15.88:0/11062921 wait complete. 2024-09-21T02:04:02.446 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 -- 172.21.15.88:0/583940400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 msgr2=0x7f8ad80cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/583940400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 0x7f8ad80cbbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005fd0 tx=0x7f8ad002a940 comp rx=0 tx=0).stop 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 -- 172.21.15.88:0/583940400 shutdown_connections 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/583940400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 0x7f8ad80cbbd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/583940400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad80ccd00 0x7f8ad815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 -- 172.21.15.88:0/583940400 >> 172.21.15.88:0/583940400 conn(0x7f8ad80cd9f0 msgr2=0x7f8ad80cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 -- 172.21.15.88:0/583940400 shutdown_connections 2024-09-21T02:04:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.558+0000 7f8ae03ad640 1 -- 172.21.15.88:0/583940400 wait complete. 2024-09-21T02:04:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.560+0000 7f8ae03ad640 1 Processor -- start 2024-09-21T02:04:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.560+0000 7f8ae03ad640 1 -- start start 2024-09-21T02:04:02.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ae03ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad80cad60 0x7f8ad81f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ae03ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 0x7f8ad8158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ae03ad640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad80ccd00 0x7f8ad8154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ae03ad640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad806d470 con 0x7f8ad80ccd00 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ae03ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad815c640 con 0x7f8ad80cb170 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ae03ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad806d5f0 con 0x7f8ad80cad60 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ade94a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad80ccd00 0x7f8ad8154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ade94a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad80ccd00 0x7f8ad8154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52536/0 (socket says 172.21.15.88:52536) 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ade94a640 1 -- 172.21.15.88:0/3379035328 learned_addr learned my addr 172.21.15.88:0/3379035328 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8ade149640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad80cad60 0x7f8ad81f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.561+0000 7f8add948640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 0x7f8ad8158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ade94a640 1 -- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad80cad60 msgr2=0x7f8ad81f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ade94a640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad80cad60 0x7f8ad81f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ade94a640 1 -- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 msgr2=0x7f8ad8158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ade94a640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 0x7f8ad8158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ade94a640 1 -- 172.21.15.88:0/3379035328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad81f5fe0 con 0x7f8ad80ccd00 2024-09-21T02:04:02.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ade94a640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad80ccd00 0x7f8ad8154600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009ca0 tx=0x7f8ad0031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8aceffd640 1 -- 172.21.15.88:0/3379035328 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0030430 con 0x7f8ad80ccd00 2024-09-21T02:04:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8aceffd640 1 -- 172.21.15.88:0/3379035328 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ad0004540 con 0x7f8ad80ccd00 2024-09-21T02:04:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad81f2260 con 0x7f8ad80ccd00 2024-09-21T02:04:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8aceffd640 1 -- 172.21.15.88:0/3379035328 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad003a440 con 0x7f8ad80ccd00 2024-09-21T02:04:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.562+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad81f2800 con 0x7f8ad80ccd00 2024-09-21T02:04:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.563+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad815f970 con 0x7f8ad80ccd00 2024-09-21T02:04:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.564+0000 7f8aceffd640 1 -- 172.21.15.88:0/3379035328 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8ad003a5e0 con 0x7f8ad80ccd00 2024-09-21T02:04:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.564+0000 7f8aceffd640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8abc078b90 0x7f8abc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.564+0000 7f8aceffd640 1 -- 172.21.15.88:0/3379035328 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8ad00bc540 con 0x7f8ad80ccd00 2024-09-21T02:04:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.564+0000 7f8ade149640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8abc078b90 0x7f8abc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.565+0000 7f8ade149640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8abc078b90 0x7f8abc07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009ac0 tx=0x7f8ad4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.569+0000 7f8aceffd640 1 -- 172.21.15.88:0/3379035328 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ad003e030 con 0x7f8ad80ccd00 2024-09-21T02:04:02.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.747+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ad8155f30 con 0x7f8ad80ccd00 2024-09-21T02:04:02.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.748+0000 7f8aceffd640 1 -- 172.21.15.88:0/3379035328 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f8ad0084aa0 con 0x7f8ad80ccd00 2024-09-21T02:04:02.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:02.748 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":21,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:59.066086+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8abc078b90 msgr2=0x7f8abc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8abc078b90 0x7f8abc07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009ac0 tx=0x7f8ad4005f90 comp rx=0 tx=0).stop 2024-09-21T02:04:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad80ccd00 msgr2=0x7f8ad8154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad80ccd00 0x7f8ad8154600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009ca0 tx=0x7f8ad0031e20 comp rx=0 tx=0).stop 2024-09-21T02:04:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 shutdown_connections 2024-09-21T02:04:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8abc078b90 0x7f8abc07b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ad80ccd00 0x7f8ad8154600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ad80cb170 0x7f8ad8158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 --2- 172.21.15.88:0/3379035328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ad80cad60 0x7f8ad81f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 >> 172.21.15.88:0/3379035328 conn(0x7f8ad80cd9f0 msgr2=0x7f8ad815eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 shutdown_connections 2024-09-21T02:04:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.750+0000 7f8ae03ad640 1 -- 172.21.15.88:0/3379035328 wait complete. 2024-09-21T02:04:02.762 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:04:02.762 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.873+0000 7fa07a220640 1 -- 172.21.15.88:0/1151748757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c48b0 msgr2=0x7fa074153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.873+0000 7fa07a220640 1 --2- 172.21.15.88:0/1151748757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c48b0 0x7fa074153730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa06c005fd0 tx=0x7fa06c02aa90 comp rx=0 tx=0).stop 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.873+0000 7fa07a220640 1 -- 172.21.15.88:0/1151748757 shutdown_connections 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.873+0000 7fa07a220640 1 --2- 172.21.15.88:0/1151748757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c48b0 0x7fa074153730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.873+0000 7fa07a220640 1 --2- 172.21.15.88:0/1151748757 >> v2:172.21.15.195:3300/0 conn(0x7fa0740c99a0 0x7fa0740712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.873+0000 7fa07a220640 1 -- 172.21.15.88:0/1151748757 >> 172.21.15.88:0/1151748757 conn(0x7fa074073210 msgr2=0x7fa0740cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.873+0000 7fa07a220640 1 -- 172.21.15.88:0/1151748757 shutdown_connections 2024-09-21T02:04:02.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.874+0000 7fa07a220640 1 -- 172.21.15.88:0/1151748757 wait complete. 2024-09-21T02:04:02.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.875+0000 7fa07a220640 1 Processor -- start 2024-09-21T02:04:02.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.875+0000 7fa07a220640 1 -- start start 2024-09-21T02:04:02.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.876+0000 7fa07a220640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0740c48b0 0x7fa0741f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.876+0000 7fa07a220640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c99a0 0x7fa07406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.876+0000 7fa07a220640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa074153320 0x7fa0741476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.876+0000 7fa07a220640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa074070dc0 con 0x7fa0740c48b0 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.876+0000 7fa07a220640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa074070ae0 con 0x7fa074153320 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.876+0000 7fa07a220640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa0740c4cc0 con 0x7fa0740c99a0 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0740c48b0 0x7fa0741f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0740c48b0 0x7fa0741f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52546/0 (socket says 172.21.15.88:52546) 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 -- 172.21.15.88:0/3860842407 learned_addr learned my addr 172.21.15.88:0/3860842407 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa073fff640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa074153320 0x7fa0741476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa072ffd640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c99a0 0x7fa07406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 -- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c99a0 msgr2=0x7fa07406f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c99a0 0x7fa07406f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 -- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa074153320 msgr2=0x7fa0741476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa074153320 0x7fa0741476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 -- 172.21.15.88:0/3860842407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0741f25f0 con 0x7fa0740c48b0 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa0737fe640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0740c48b0 0x7fa0741f20b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e9f0 tx=0x7fa06400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa05bfff640 1 -- 172.21.15.88:0/3860842407 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400cd80 con 0x7fa0740c48b0 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa05bfff640 1 -- 172.21.15.88:0/3860842407 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa064004020 con 0x7fa0740c48b0 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0741f5350 con 0x7fa0740c48b0 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa05bfff640 1 -- 172.21.15.88:0/3860842407 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064004470 con 0x7fa0740c48b0 2024-09-21T02:04:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.877+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0741f5610 con 0x7fa0740c48b0 2024-09-21T02:04:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.878+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0740ca8c0 con 0x7fa0740c48b0 2024-09-21T02:04:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.879+0000 7fa05bfff640 1 -- 172.21.15.88:0/3860842407 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa0640035f0 con 0x7fa0740c48b0 2024-09-21T02:04:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.880+0000 7fa05bfff640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa060078b90 0x7fa06007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.880+0000 7fa05bfff640 1 -- 172.21.15.88:0/3860842407 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa06409b5a0 con 0x7fa0740c48b0 2024-09-21T02:04:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.880+0000 7fa072ffd640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa060078b90 0x7fa06007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.880+0000 7fa072ffd640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa060078b90 0x7fa06007b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa07415a330 tx=0x7fa068009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:02.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:02.884+0000 7fa05bfff640 1 -- 172.21.15.88:0/3860842407 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0640633a0 con 0x7fa0740c48b0 2024-09-21T02:04:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.057+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0740cb660 con 0x7fa060078b90 2024-09-21T02:04:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.062+0000 7fa05bfff640 1 -- 172.21.15.88:0/3860842407 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fa0740cb660 con 0x7fa060078b90 2024-09-21T02:04:03.065 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:04:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa060078b90 msgr2=0x7fa06007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa060078b90 0x7fa06007b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa07415a330 tx=0x7fa068009040 comp rx=0 tx=0).stop 2024-09-21T02:04:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0740c48b0 msgr2=0x7fa0741f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0740c48b0 0x7fa0741f20b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e9f0 tx=0x7fa06400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 shutdown_connections 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa060078b90 0x7fa06007b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa074153320 0x7fa0741476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0740c99a0 0x7fa07406f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 --2- 172.21.15.88:0/3860842407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0740c48b0 0x7fa0741f20b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.067+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 >> 172.21.15.88:0/3860842407 conn(0x7fa074073210 msgr2=0x7fa0740c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.068+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 shutdown_connections 2024-09-21T02:04:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.068+0000 7fa07a220640 1 -- 172.21.15.88:0/3860842407 wait complete. 2024-09-21T02:04:03.081 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T02:04:02.819070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239836,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734884,"statfs":{"total":773094113280,"available":772848521216,"internally_reserved":0,"allocated":2834432,"data_stored":1750112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001559"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074727+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:03:59.074727+0000","last_peered":"2024-09-21T02:03:59.074727+0000","last_clean":"2024-09-21T02:03:59.074727+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:03:59.074727+0000","last_undegraded":"2024-09-21T02:03:59.074727+0000","last_fullsized":"2024-09-21T02:03:59.074727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073213+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:03:59.073213+0000","last_peered":"2024-09-21T02:03:59.073213+0000","last_clean":"2024-09-21T02:03:59.073213+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:03:59.073213+0000","last_undegraded":"2024-09-21T02:03:59.073213+0000","last_fullsized":"2024-09-21T02:03:59.073213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085286+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:03:58.085286+0000","last_peered":"2024-09-21T02:03:58.085286+0000","last_clean":"2024-09-21T02:03:58.085286+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:03:58.085286+0000","last_undegraded":"2024-09-21T02:03:58.085286+0000","last_fullsized":"2024-09-21T02:03:58.085286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:17.155275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075220+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:03:59.075220+0000","last_peered":"2024-09-21T02:03:59.075220+0000","last_clean":"2024-09-21T02:03:59.075220+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:03:59.075220+0000","last_undegraded":"2024-09-21T02:03:59.075220+0000","last_fullsized":"2024-09-21T02:03:59.075220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074990+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:03:59.074990+0000","last_peered":"2024-09-21T02:03:59.074990+0000","last_clean":"2024-09-21T02:03:59.074990+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:03:59.074990+0000","last_undegraded":"2024-09-21T02:03:59.074990+0000","last_fullsized":"2024-09-21T02:03:59.074990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074223+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:03:59.074223+0000","last_peered":"2024-09-21T02:03:59.074223+0000","last_clean":"2024-09-21T02:03:59.074223+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:03:59.074223+0000","last_undegraded":"2024-09-21T02:03:59.074223+0000","last_fullsized":"2024-09-21T02:03:59.074223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074095+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:03:59.074095+0000","last_peered":"2024-09-21T02:03:59.074095+0000","last_clean":"2024-09-21T02:03:59.074095+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:03:59.074095+0000","last_undegraded":"2024-09-21T02:03:59.074095+0000","last_fullsized":"2024-09-21T02:03:59.074095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835347+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:03:59.835347+0000","last_peered":"2024-09-21T02:03:59.835347+0000","last_clean":"2024-09-21T02:03:59.835347+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:03:59.835347+0000","last_undegraded":"2024-09-21T02:03:59.835347+0000","last_fullsized":"2024-09-21T02:03:59.835347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074637+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:03:59.074637+0000","last_peered":"2024-09-21T02:03:59.074637+0000","last_clean":"2024-09-21T02:03:59.074637+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:03:59.074637+0000","last_undegraded":"2024-09-21T02:03:59.074637+0000","last_fullsized":"2024-09-21T02:03:59.074637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.087515+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:03:58.087515+0000","last_peered":"2024-09-21T02:03:58.087515+0000","last_clean":"2024-09-21T02:03:58.087515+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:03:58.087515+0000","last_undegraded":"2024-09-21T02:03:58.087515+0000","last_fullsized":"2024-09-21T02:03:58.087515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:45.822704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.086382+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:03:58.086382+0000","last_peered":"2024-09-21T02:03:58.086382+0000","last_clean":"2024-09-21T02:03:58.086382+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:03:58.086382+0000","last_undegraded":"2024-09-21T02:03:58.086382+0000","last_fullsized":"2024-09-21T02:03:58.086382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:41.786759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781637+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:03:59.781637+0000","last_peered":"2024-09-21T02:03:59.781637+0000","last_clean":"2024-09-21T02:03:59.781637+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:03:59.781637+0000","last_undegraded":"2024-09-21T02:03:59.781637+0000","last_fullsized":"2024-09-21T02:03:59.781637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835918+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:03:59.835918+0000","last_peered":"2024-09-21T02:03:59.835918+0000","last_clean":"2024-09-21T02:03:59.835918+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:03:59.835918+0000","last_undegraded":"2024-09-21T02:03:59.835918+0000","last_fullsized":"2024-09-21T02:03:59.835918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.084346+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:03:58.084346+0000","last_peered":"2024-09-21T02:03:58.084346+0000","last_clean":"2024-09-21T02:03:58.084346+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:03:58.084346+0000","last_undegraded":"2024-09-21T02:03:58.084346+0000","last_fullsized":"2024-09-21T02:03:58.084346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:12.494411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072763+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:03:59.072763+0000","last_peered":"2024-09-21T02:03:59.072763+0000","last_clean":"2024-09-21T02:03:59.072763+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:03:59.072763+0000","last_undegraded":"2024-09-21T02:03:59.072763+0000","last_fullsized":"2024-09-21T02:03:59.072763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075548+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:03:59.075548+0000","last_peered":"2024-09-21T02:03:59.075548+0000","last_clean":"2024-09-21T02:03:59.075548+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:03:59.075548+0000","last_undegraded":"2024-09-21T02:03:59.075548+0000","last_fullsized":"2024-09-21T02:03:59.075548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075910+0000","last_change":"2024-09-21T02:03:58.074570+0000","last_active":"2024-09-21T02:03:59.075910+0000","last_peered":"2024-09-21T02:03:59.075910+0000","last_clean":"2024-09-21T02:03:59.075910+0000","last_became_active":"2024-09-21T02:03:58.074271+0000","last_became_peered":"2024-09-21T02:03:58.074271+0000","last_unstale":"2024-09-21T02:03:59.075910+0000","last_undegraded":"2024-09-21T02:03:59.075910+0000","last_fullsized":"2024-09-21T02:03:59.075910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.292792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835423+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:03:59.835423+0000","last_peered":"2024-09-21T02:03:59.835423+0000","last_clean":"2024-09-21T02:03:59.835423+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:03:59.835423+0000","last_undegraded":"2024-09-21T02:03:59.835423+0000","last_fullsized":"2024-09-21T02:03:59.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076132+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:03:59.076132+0000","last_peered":"2024-09-21T02:03:59.076132+0000","last_clean":"2024-09-21T02:03:59.076132+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:03:59.076132+0000","last_undegraded":"2024-09-21T02:03:59.076132+0000","last_fullsized":"2024-09-21T02:03:59.076132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.089850+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:03:58.089850+0000","last_peered":"2024-09-21T02:03:58.089850+0000","last_clean":"2024-09-21T02:03:58.089850+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:03:58.089850+0000","last_undegraded":"2024-09-21T02:03:58.089850+0000","last_fullsized":"2024-09-21T02:03:58.089850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:04.159342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085783+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:03:58.085783+0000","last_peered":"2024-09-21T02:03:58.085783+0000","last_clean":"2024-09-21T02:03:58.085783+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:03:58.085783+0000","last_undegraded":"2024-09-21T02:03:58.085783+0000","last_fullsized":"2024-09-21T02:03:58.085783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:16.364344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075251+0000","last_change":"2024-09-21T02:03:58.076378+0000","last_active":"2024-09-21T02:03:59.075251+0000","last_peered":"2024-09-21T02:03:59.075251+0000","last_clean":"2024-09-21T02:03:59.075251+0000","last_became_active":"2024-09-21T02:03:58.076075+0000","last_became_peered":"2024-09-21T02:03:58.076075+0000","last_unstale":"2024-09-21T02:03:59.075251+0000","last_undegraded":"2024-09-21T02:03:59.075251+0000","last_fullsized":"2024-09-21T02:03:59.075251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:54.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835596+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:03:59.835596+0000","last_peered":"2024-09-21T02:03:59.835596+0000","last_clean":"2024-09-21T02:03:59.835596+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:03:59.835596+0000","last_undegraded":"2024-09-21T02:03:59.835596+0000","last_fullsized":"2024-09-21T02:03:59.835596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074571+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:03:59.074571+0000","last_peered":"2024-09-21T02:03:59.074571+0000","last_clean":"2024-09-21T02:03:59.074571+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:03:59.074571+0000","last_undegraded":"2024-09-21T02:03:59.074571+0000","last_fullsized":"2024-09-21T02:03:59.074571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836762+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:03:59.836762+0000","last_peered":"2024-09-21T02:03:59.836762+0000","last_clean":"2024-09-21T02:03:59.836762+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:03:59.836762+0000","last_undegraded":"2024-09-21T02:03:59.836762+0000","last_fullsized":"2024-09-21T02:03:59.836762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085813+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:03:58.085813+0000","last_peered":"2024-09-21T02:03:58.085813+0000","last_clean":"2024-09-21T02:03:58.085813+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:03:58.085813+0000","last_undegraded":"2024-09-21T02:03:58.085813+0000","last_fullsized":"2024-09-21T02:03:58.085813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:41.739565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073053+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:03:59.073053+0000","last_peered":"2024-09-21T02:03:59.073053+0000","last_clean":"2024-09-21T02:03:59.073053+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:03:59.073053+0000","last_undegraded":"2024-09-21T02:03:59.073053+0000","last_fullsized":"2024-09-21T02:03:59.073053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836383+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:03:59.836383+0000","last_peered":"2024-09-21T02:03:59.836383+0000","last_clean":"2024-09-21T02:03:59.836383+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:03:59.836383+0000","last_undegraded":"2024-09-21T02:03:59.836383+0000","last_fullsized":"2024-09-21T02:03:59.836383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085757+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:03:58.085757+0000","last_peered":"2024-09-21T02:03:58.085757+0000","last_clean":"2024-09-21T02:03:58.085757+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:03:58.085757+0000","last_undegraded":"2024-09-21T02:03:58.085757+0000","last_fullsized":"2024-09-21T02:03:58.085757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:43.473957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075554+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:03:59.075554+0000","last_peered":"2024-09-21T02:03:59.075554+0000","last_clean":"2024-09-21T02:03:59.075554+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:03:59.075554+0000","last_undegraded":"2024-09-21T02:03:59.075554+0000","last_fullsized":"2024-09-21T02:03:59.075554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835391+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:03:59.835391+0000","last_peered":"2024-09-21T02:03:59.835391+0000","last_clean":"2024-09-21T02:03:59.835391+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:03:59.835391+0000","last_undegraded":"2024-09-21T02:03:59.835391+0000","last_fullsized":"2024-09-21T02:03:59.835391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.087474+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:03:58.087474+0000","last_peered":"2024-09-21T02:03:58.087474+0000","last_clean":"2024-09-21T02:03:58.087474+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:03:58.087474+0000","last_undegraded":"2024-09-21T02:03:58.087474+0000","last_fullsized":"2024-09-21T02:03:58.087474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:11.729550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781525+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:03:59.781525+0000","last_peered":"2024-09-21T02:03:59.781525+0000","last_clean":"2024-09-21T02:03:59.781525+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:03:59.781525+0000","last_undegraded":"2024-09-21T02:03:59.781525+0000","last_fullsized":"2024-09-21T02:03:59.781525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836683+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:03:59.836683+0000","last_peered":"2024-09-21T02:03:59.836683+0000","last_clean":"2024-09-21T02:03:59.836683+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:03:59.836683+0000","last_undegraded":"2024-09-21T02:03:59.836683+0000","last_fullsized":"2024-09-21T02:03:59.836683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085712+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:03:58.085712+0000","last_peered":"2024-09-21T02:03:58.085712+0000","last_clean":"2024-09-21T02:03:58.085712+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:03:58.085712+0000","last_undegraded":"2024-09-21T02:03:58.085712+0000","last_fullsized":"2024-09-21T02:03:58.085712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:46.972262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.067310+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:03:58.067310+0000","last_peered":"2024-09-21T02:03:58.067310+0000","last_clean":"2024-09-21T02:03:58.067310+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:03:58.067310+0000","last_undegraded":"2024-09-21T02:03:58.067310+0000","last_fullsized":"2024-09-21T02:03:58.067310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835432+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:03:59.835432+0000","last_peered":"2024-09-21T02:03:59.835432+0000","last_clean":"2024-09-21T02:03:59.835432+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:03:59.835432+0000","last_undegraded":"2024-09-21T02:03:59.835432+0000","last_fullsized":"2024-09-21T02:03:59.835432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835573+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:03:59.835573+0000","last_peered":"2024-09-21T02:03:59.835573+0000","last_clean":"2024-09-21T02:03:59.835573+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:03:59.835573+0000","last_undegraded":"2024-09-21T02:03:59.835573+0000","last_fullsized":"2024-09-21T02:03:59.835573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074657+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:03:59.074657+0000","last_peered":"2024-09-21T02:03:59.074657+0000","last_clean":"2024-09-21T02:03:59.074657+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:03:59.074657+0000","last_undegraded":"2024-09-21T02:03:59.074657+0000","last_fullsized":"2024-09-21T02:03:59.074657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072701+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:03:59.072701+0000","last_peered":"2024-09-21T02:03:59.072701+0000","last_clean":"2024-09-21T02:03:59.072701+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:03:59.072701+0000","last_undegraded":"2024-09-21T02:03:59.072701+0000","last_fullsized":"2024-09-21T02:03:59.072701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.066693+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:03:58.066693+0000","last_peered":"2024-09-21T02:03:58.066693+0000","last_clean":"2024-09-21T02:03:58.066693+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:03:58.066693+0000","last_undegraded":"2024-09-21T02:03:58.066693+0000","last_fullsized":"2024-09-21T02:03:58.066693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074748+0000","last_change":"2024-09-21T02:03:58.075434+0000","last_active":"2024-09-21T02:03:59.074748+0000","last_peered":"2024-09-21T02:03:59.074748+0000","last_clean":"2024-09-21T02:03:59.074748+0000","last_became_active":"2024-09-21T02:03:58.075130+0000","last_became_peered":"2024-09-21T02:03:58.075130+0000","last_unstale":"2024-09-21T02:03:59.074748+0000","last_undegraded":"2024-09-21T02:03:59.074748+0000","last_fullsized":"2024-09-21T02:03:59.074748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:56.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074818+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:03:59.074818+0000","last_peered":"2024-09-21T02:03:59.074818+0000","last_clean":"2024-09-21T02:03:59.074818+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:03:59.074818+0000","last_undegraded":"2024-09-21T02:03:59.074818+0000","last_fullsized":"2024-09-21T02:03:59.074818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085666+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:03:58.085666+0000","last_peered":"2024-09-21T02:03:58.085666+0000","last_clean":"2024-09-21T02:03:58.085666+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:03:58.085666+0000","last_undegraded":"2024-09-21T02:03:58.085666+0000","last_fullsized":"2024-09-21T02:03:58.085666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:30.459067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072952+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:03:59.072952+0000","last_peered":"2024-09-21T02:03:59.072952+0000","last_clean":"2024-09-21T02:03:59.072952+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:03:59.072952+0000","last_undegraded":"2024-09-21T02:03:59.072952+0000","last_fullsized":"2024-09-21T02:03:59.072952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781695+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:03:59.781695+0000","last_peered":"2024-09-21T02:03:59.781695+0000","last_clean":"2024-09-21T02:03:59.781695+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:03:59.781695+0000","last_undegraded":"2024-09-21T02:03:59.781695+0000","last_fullsized":"2024-09-21T02:03:59.781695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.067848+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:03:58.067848+0000","last_peered":"2024-09-21T02:03:58.067848+0000","last_clean":"2024-09-21T02:03:58.067848+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:03:58.067848+0000","last_undegraded":"2024-09-21T02:03:58.067848+0000","last_fullsized":"2024-09-21T02:03:58.067848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075568+0000","last_change":"2024-09-21T02:03:58.080947+0000","last_active":"2024-09-21T02:03:59.075568+0000","last_peered":"2024-09-21T02:03:59.075568+0000","last_clean":"2024-09-21T02:03:59.075568+0000","last_became_active":"2024-09-21T02:03:58.080632+0000","last_became_peered":"2024-09-21T02:03:58.080632+0000","last_unstale":"2024-09-21T02:03:59.075568+0000","last_undegraded":"2024-09-21T02:03:59.075568+0000","last_fullsized":"2024-09-21T02:03:59.075568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:40.307689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.068339+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:03:58.068339+0000","last_peered":"2024-09-21T02:03:58.068339+0000","last_clean":"2024-09-21T02:03:58.068339+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:03:58.068339+0000","last_undegraded":"2024-09-21T02:03:58.068339+0000","last_fullsized":"2024-09-21T02:03:58.068339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085448+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:03:58.085448+0000","last_peered":"2024-09-21T02:03:58.085448+0000","last_clean":"2024-09-21T02:03:58.085448+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:03:58.085448+0000","last_undegraded":"2024-09-21T02:03:58.085448+0000","last_fullsized":"2024-09-21T02:03:58.085448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:22.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075336+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:03:59.075336+0000","last_peered":"2024-09-21T02:03:59.075336+0000","last_clean":"2024-09-21T02:03:59.075336+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:03:59.075336+0000","last_undegraded":"2024-09-21T02:03:59.075336+0000","last_fullsized":"2024-09-21T02:03:59.075336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075210+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:03:59.075210+0000","last_peered":"2024-09-21T02:03:59.075210+0000","last_clean":"2024-09-21T02:03:59.075210+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:03:59.075210+0000","last_undegraded":"2024-09-21T02:03:59.075210+0000","last_fullsized":"2024-09-21T02:03:59.075210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.066464+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:03:58.066464+0000","last_peered":"2024-09-21T02:03:58.066464+0000","last_clean":"2024-09-21T02:03:58.066464+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:03:58.066464+0000","last_undegraded":"2024-09-21T02:03:58.066464+0000","last_fullsized":"2024-09-21T02:03:58.066464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.086424+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:03:58.086424+0000","last_peered":"2024-09-21T02:03:58.086424+0000","last_clean":"2024-09-21T02:03:58.086424+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:03:58.086424+0000","last_undegraded":"2024-09-21T02:03:58.086424+0000","last_fullsized":"2024-09-21T02:03:58.086424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:10.961713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835098+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:03:59.835098+0000","last_peered":"2024-09-21T02:03:59.835098+0000","last_clean":"2024-09-21T02:03:59.835098+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:03:59.835098+0000","last_undegraded":"2024-09-21T02:03:59.835098+0000","last_fullsized":"2024-09-21T02:03:59.835098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074582+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:03:59.074582+0000","last_peered":"2024-09-21T02:03:59.074582+0000","last_clean":"2024-09-21T02:03:59.074582+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:03:59.074582+0000","last_undegraded":"2024-09-21T02:03:59.074582+0000","last_fullsized":"2024-09-21T02:03:59.074582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781506+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:03:59.781506+0000","last_peered":"2024-09-21T02:03:59.781506+0000","last_clean":"2024-09-21T02:03:59.781506+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:03:59.781506+0000","last_undegraded":"2024-09-21T02:03:59.781506+0000","last_fullsized":"2024-09-21T02:03:59.781506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074555+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:03:59.074555+0000","last_peered":"2024-09-21T02:03:59.074555+0000","last_clean":"2024-09-21T02:03:59.074555+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:03:59.074555+0000","last_undegraded":"2024-09-21T02:03:59.074555+0000","last_fullsized":"2024-09-21T02:03:59.074555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835494+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:03:59.835494+0000","last_peered":"2024-09-21T02:03:59.835494+0000","last_clean":"2024-09-21T02:03:59.835494+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:03:59.835494+0000","last_undegraded":"2024-09-21T02:03:59.835494+0000","last_fullsized":"2024-09-21T02:03:59.835494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074911+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:03:59.074911+0000","last_peered":"2024-09-21T02:03:59.074911+0000","last_clean":"2024-09-21T02:03:59.074911+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:03:59.074911+0000","last_undegraded":"2024-09-21T02:03:59.074911+0000","last_fullsized":"2024-09-21T02:03:59.074911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073063+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:03:59.073063+0000","last_peered":"2024-09-21T02:03:59.073063+0000","last_clean":"2024-09-21T02:03:59.073063+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:03:59.073063+0000","last_undegraded":"2024-09-21T02:03:59.073063+0000","last_fullsized":"2024-09-21T02:03:59.073063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075681+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:03:59.075681+0000","last_peered":"2024-09-21T02:03:59.075681+0000","last_clean":"2024-09-21T02:03:59.075681+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:03:59.075681+0000","last_undegraded":"2024-09-21T02:03:59.075681+0000","last_fullsized":"2024-09-21T02:03:59.075681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835624+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:03:59.835624+0000","last_peered":"2024-09-21T02:03:59.835624+0000","last_clean":"2024-09-21T02:03:59.835624+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:03:59.835624+0000","last_undegraded":"2024-09-21T02:03:59.835624+0000","last_fullsized":"2024-09-21T02:03:59.835624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075817+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:03:59.075817+0000","last_peered":"2024-09-21T02:03:59.075817+0000","last_clean":"2024-09-21T02:03:59.075817+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:03:59.075817+0000","last_undegraded":"2024-09-21T02:03:59.075817+0000","last_fullsized":"2024-09-21T02:03:59.075817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074041+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:03:59.074041+0000","last_peered":"2024-09-21T02:03:59.074041+0000","last_clean":"2024-09-21T02:03:59.074041+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:03:59.074041+0000","last_undegraded":"2024-09-21T02:03:59.074041+0000","last_fullsized":"2024-09-21T02:03:59.074041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074376+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:03:59.074376+0000","last_peered":"2024-09-21T02:03:59.074376+0000","last_clean":"2024-09-21T02:03:59.074376+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:03:59.074376+0000","last_undegraded":"2024-09-21T02:03:59.074376+0000","last_fullsized":"2024-09-21T02:03:59.074376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835923+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:03:59.835923+0000","last_peered":"2024-09-21T02:03:59.835923+0000","last_clean":"2024-09-21T02:03:59.835923+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:03:59.835923+0000","last_undegraded":"2024-09-21T02:03:59.835923+0000","last_fullsized":"2024-09-21T02:03:59.835923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073892+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:03:59.073892+0000","last_peered":"2024-09-21T02:03:59.073892+0000","last_clean":"2024-09-21T02:03:59.073892+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:03:59.073892+0000","last_undegraded":"2024-09-21T02:03:59.073892+0000","last_fullsized":"2024-09-21T02:03:59.073892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085608+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:03:58.085608+0000","last_peered":"2024-09-21T02:03:58.085608+0000","last_clean":"2024-09-21T02:03:58.085608+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:03:58.085608+0000","last_undegraded":"2024-09-21T02:03:58.085608+0000","last_fullsized":"2024-09-21T02:03:58.085608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:40.368766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835266+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:03:59.835266+0000","last_peered":"2024-09-21T02:03:59.835266+0000","last_clean":"2024-09-21T02:03:59.835266+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:03:59.835266+0000","last_undegraded":"2024-09-21T02:03:59.835266+0000","last_fullsized":"2024-09-21T02:03:59.835266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072781+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:03:59.072781+0000","last_peered":"2024-09-21T02:03:59.072781+0000","last_clean":"2024-09-21T02:03:59.072781+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:03:59.072781+0000","last_undegraded":"2024-09-21T02:03:59.072781+0000","last_fullsized":"2024-09-21T02:03:59.072781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075493+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:03:59.075493+0000","last_peered":"2024-09-21T02:03:59.075493+0000","last_clean":"2024-09-21T02:03:59.075493+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:03:59.075493+0000","last_undegraded":"2024-09-21T02:03:59.075493+0000","last_fullsized":"2024-09-21T02:03:59.075493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781505+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:03:59.781505+0000","last_peered":"2024-09-21T02:03:59.781505+0000","last_clean":"2024-09-21T02:03:59.781505+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:03:59.781505+0000","last_undegraded":"2024-09-21T02:03:59.781505+0000","last_fullsized":"2024-09-21T02:03:59.781505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072896+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:03:59.072896+0000","last_peered":"2024-09-21T02:03:59.072896+0000","last_clean":"2024-09-21T02:03:59.072896+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:03:59.072896+0000","last_undegraded":"2024-09-21T02:03:59.072896+0000","last_fullsized":"2024-09-21T02:03:59.072896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076487+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:03:59.076487+0000","last_peered":"2024-09-21T02:03:59.076487+0000","last_clean":"2024-09-21T02:03:59.076487+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:03:59.076487+0000","last_undegraded":"2024-09-21T02:03:59.076487+0000","last_fullsized":"2024-09-21T02:03:59.076487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075934+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:03:59.075934+0000","last_peered":"2024-09-21T02:03:59.075934+0000","last_clean":"2024-09-21T02:03:59.075934+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:03:59.075934+0000","last_undegraded":"2024-09-21T02:03:59.075934+0000","last_fullsized":"2024-09-21T02:03:59.075934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073021+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:03:59.073021+0000","last_peered":"2024-09-21T02:03:59.073021+0000","last_clean":"2024-09-21T02:03:59.073021+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:03:59.073021+0000","last_undegraded":"2024-09-21T02:03:59.073021+0000","last_fullsized":"2024-09-21T02:03:59.073021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073068+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:03:59.073068+0000","last_peered":"2024-09-21T02:03:59.073068+0000","last_clean":"2024-09-21T02:03:59.073068+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:03:59.073068+0000","last_undegraded":"2024-09-21T02:03:59.073068+0000","last_fullsized":"2024-09-21T02:03:59.073068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836949+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:03:59.836949+0000","last_peered":"2024-09-21T02:03:59.836949+0000","last_clean":"2024-09-21T02:03:59.836949+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:03:59.836949+0000","last_undegraded":"2024-09-21T02:03:59.836949+0000","last_fullsized":"2024-09-21T02:03:59.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.066772+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:03:58.066772+0000","last_peered":"2024-09-21T02:03:58.066772+0000","last_clean":"2024-09-21T02:03:58.066772+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:03:58.066772+0000","last_undegraded":"2024-09-21T02:03:58.066772+0000","last_fullsized":"2024-09-21T02:03:58.066772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085374+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:03:58.085374+0000","last_peered":"2024-09-21T02:03:58.085374+0000","last_clean":"2024-09-21T02:03:58.085374+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:03:58.085374+0000","last_undegraded":"2024-09-21T02:03:58.085374+0000","last_fullsized":"2024-09-21T02:03:58.085374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:46.637904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073699+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:03:59.073699+0000","last_peered":"2024-09-21T02:03:59.073699+0000","last_clean":"2024-09-21T02:03:59.073699+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:03:59.073699+0000","last_undegraded":"2024-09-21T02:03:59.073699+0000","last_fullsized":"2024-09-21T02:03:59.073699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781583+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:03:59.781583+0000","last_peered":"2024-09-21T02:03:59.781583+0000","last_clean":"2024-09-21T02:03:59.781583+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:03:59.781583+0000","last_undegraded":"2024-09-21T02:03:59.781583+0000","last_fullsized":"2024-09-21T02:03:59.781583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072755+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:03:59.072755+0000","last_peered":"2024-09-21T02:03:59.072755+0000","last_clean":"2024-09-21T02:03:59.072755+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:03:59.072755+0000","last_undegraded":"2024-09-21T02:03:59.072755+0000","last_fullsized":"2024-09-21T02:03:59.072755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835669+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:03:59.835669+0000","last_peered":"2024-09-21T02:03:59.835669+0000","last_clean":"2024-09-21T02:03:59.835669+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:03:59.835669+0000","last_undegraded":"2024-09-21T02:03:59.835669+0000","last_fullsized":"2024-09-21T02:03:59.835669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072863+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:03:59.072863+0000","last_peered":"2024-09-21T02:03:59.072863+0000","last_clean":"2024-09-21T02:03:59.072863+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:03:59.072863+0000","last_undegraded":"2024-09-21T02:03:59.072863+0000","last_fullsized":"2024-09-21T02:03:59.072863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836615+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:03:59.836615+0000","last_peered":"2024-09-21T02:03:59.836615+0000","last_clean":"2024-09-21T02:03:59.836615+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:03:59.836615+0000","last_undegraded":"2024-09-21T02:03:59.836615+0000","last_fullsized":"2024-09-21T02:03:59.836615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075290+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:03:59.075290+0000","last_peered":"2024-09-21T02:03:59.075290+0000","last_clean":"2024-09-21T02:03:59.075290+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:03:59.075290+0000","last_undegraded":"2024-09-21T02:03:59.075290+0000","last_fullsized":"2024-09-21T02:03:59.075290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.084565+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:03:58.084565+0000","last_peered":"2024-09-21T02:03:58.084565+0000","last_clean":"2024-09-21T02:03:58.084565+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:03:58.084565+0000","last_undegraded":"2024-09-21T02:03:58.084565+0000","last_fullsized":"2024-09-21T02:03:58.084565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:08.508247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076421+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:03:59.076421+0000","last_peered":"2024-09-21T02:03:59.076421+0000","last_clean":"2024-09-21T02:03:59.076421+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:03:59.076421+0000","last_undegraded":"2024-09-21T02:03:59.076421+0000","last_fullsized":"2024-09-21T02:03:59.076421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074280+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:03:59.074280+0000","last_peered":"2024-09-21T02:03:59.074280+0000","last_clean":"2024-09-21T02:03:59.074280+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:03:59.074280+0000","last_undegraded":"2024-09-21T02:03:59.074280+0000","last_fullsized":"2024-09-21T02:03:59.074280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075820+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:03:59.075820+0000","last_peered":"2024-09-21T02:03:59.075820+0000","last_clean":"2024-09-21T02:03:59.075820+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:03:59.075820+0000","last_undegraded":"2024-09-21T02:03:59.075820+0000","last_fullsized":"2024-09-21T02:03:59.075820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075758+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:03:59.075758+0000","last_peered":"2024-09-21T02:03:59.075758+0000","last_clean":"2024-09-21T02:03:59.075758+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:03:59.075758+0000","last_undegraded":"2024-09-21T02:03:59.075758+0000","last_fullsized":"2024-09-21T02:03:59.075758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085611+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:03:58.085611+0000","last_peered":"2024-09-21T02:03:58.085611+0000","last_clean":"2024-09-21T02:03:58.085611+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:03:58.085611+0000","last_undegraded":"2024-09-21T02:03:58.085611+0000","last_fullsized":"2024-09-21T02:03:58.085611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:56.367906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837047+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:03:59.837047+0000","last_peered":"2024-09-21T02:03:59.837047+0000","last_clean":"2024-09-21T02:03:59.837047+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:03:59.837047+0000","last_undegraded":"2024-09-21T02:03:59.837047+0000","last_fullsized":"2024-09-21T02:03:59.837047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837178+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:03:59.837178+0000","last_peered":"2024-09-21T02:03:59.837178+0000","last_clean":"2024-09-21T02:03:59.837178+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:03:59.837178+0000","last_undegraded":"2024-09-21T02:03:59.837178+0000","last_fullsized":"2024-09-21T02:03:59.837178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.067826+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:03:58.067826+0000","last_peered":"2024-09-21T02:03:58.067826+0000","last_clean":"2024-09-21T02:03:58.067826+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:03:58.067826+0000","last_undegraded":"2024-09-21T02:03:58.067826+0000","last_fullsized":"2024-09-21T02:03:58.067826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":802816,"data_stored":663115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35548,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336292,"statfs":{"total":96636764160,"available":96600363008,"internally_reserved":0,"allocated":196608,"data_stored":66447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":656815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:03.085 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 -- 172.21.15.88:0/2506847420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cb170 msgr2=0x7f63dc0cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 --2- 172.21.15.88:0/2506847420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cb170 0x7f63dc0cbbd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f63dc06d9f0 tx=0x7f63d402aa70 comp rx=0 tx=0).stop 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 -- 172.21.15.88:0/2506847420 shutdown_connections 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 --2- 172.21.15.88:0/2506847420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cb170 0x7f63dc0cbbd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 --2- 172.21.15.88:0/2506847420 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63dc0ccd00 0x7f63dc15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 -- 172.21.15.88:0/2506847420 >> 172.21.15.88:0/2506847420 conn(0x7f63dc0cd9f0 msgr2=0x7f63dc0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 -- 172.21.15.88:0/2506847420 shutdown_connections 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.233+0000 7f63e43a9640 1 -- 172.21.15.88:0/2506847420 wait complete. 2024-09-21T02:04:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.235+0000 7f63e43a9640 1 Processor -- start 2024-09-21T02:04:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.235+0000 7f63e43a9640 1 -- start start 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e43a9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cad60 0x7f63dc0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e43a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63dc0cb170 0x7f63dc1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e43a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63dc0ccd00 0x7f63dc1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e43a9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc06d470 con 0x7f63dc0cad60 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e43a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc15c640 con 0x7f63dc0ccd00 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e43a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc06d5f0 con 0x7f63dc0cb170 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cad60 0x7f63dc0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cad60 0x7f63dc0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52556/0 (socket says 172.21.15.88:52556) 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 -- 172.21.15.88:0/464010705 learned_addr learned my addr 172.21.15.88:0/464010705 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 -- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63dc0cb170 msgr2=0x7f63dc1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63dc0cb170 0x7f63dc1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 -- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63dc0ccd00 msgr2=0x7f63dc1536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63dc0ccd00 0x7f63dc1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 -- 172.21.15.88:0/464010705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc0d0cf0 con 0x7f63dc0cad60 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e2145640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cad60 0x7f63dc0d00b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c9a0 tx=0x7f63d800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63d2ffd640 1 -- 172.21.15.88:0/464010705 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8007bb0 con 0x7f63dc0cad60 2024-09-21T02:04:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63d2ffd640 1 -- 172.21.15.88:0/464010705 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63d8007e40 con 0x7f63dc0cad60 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.236+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc157610 con 0x7f63dc0cad60 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.237+0000 7f63d2ffd640 1 -- 172.21.15.88:0/464010705 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d800f690 con 0x7f63dc0cad60 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.237+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc1e65b0 con 0x7f63dc0cad60 2024-09-21T02:04:03.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.237+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc06ddb0 con 0x7f63dc0cad60 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.238+0000 7f63d2ffd640 1 -- 172.21.15.88:0/464010705 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63d800f830 con 0x7f63dc0cad60 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.239+0000 7f63d2ffd640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f63c0078bc0 0x7f63c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.239+0000 7f63d2ffd640 1 -- 172.21.15.88:0/464010705 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f63d809b610 con 0x7f63dc0cad60 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.239+0000 7f63e1944640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f63c0078bc0 0x7f63c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.240+0000 7f63e1944640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f63c0078bc0 0x7f63c007b0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006fd0 tx=0x7f63cc006bc0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.244+0000 7f63d2ffd640 1 -- 172.21.15.88:0/464010705 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63d8014030 con 0x7f63dc0cad60 2024-09-21T02:04:03.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.419+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63dc155f30 con 0x7f63c0078bc0 2024-09-21T02:04:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.424+0000 7f63d2ffd640 1 -- 172.21.15.88:0/464010705 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f63dc155f30 con 0x7f63c0078bc0 2024-09-21T02:04:03.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:03.427 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:04:03.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f63c0078bc0 msgr2=0x7f63c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f63c0078bc0 0x7f63c007b0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006fd0 tx=0x7f63cc006bc0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cad60 msgr2=0x7f63dc0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cad60 0x7f63dc0d00b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c9a0 tx=0x7f63d800ce60 comp rx=0 tx=0).stop 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 shutdown_connections 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f63c0078bc0 0x7f63c007b0d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63dc0ccd00 0x7f63dc1536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63dc0cb170 0x7f63dc1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 --2- 172.21.15.88:0/464010705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63dc0cad60 0x7f63dc0d00b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 >> 172.21.15.88:0/464010705 conn(0x7f63dc0cd9f0 msgr2=0x7f63dc0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 shutdown_connections 2024-09-21T02:04:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.427+0000 7f63e43a9640 1 -- 172.21.15.88:0/464010705 wait complete. 2024-09-21T02:04:03.438 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T02:04:02.819070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239836,"kb_used_data":2768,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734884,"statfs":{"total":773094113280,"available":772848521216,"internally_reserved":0,"allocated":2834432,"data_stored":1750112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001559"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074727+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:03:59.074727+0000","last_peered":"2024-09-21T02:03:59.074727+0000","last_clean":"2024-09-21T02:03:59.074727+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:03:59.074727+0000","last_undegraded":"2024-09-21T02:03:59.074727+0000","last_fullsized":"2024-09-21T02:03:59.074727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073213+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:03:59.073213+0000","last_peered":"2024-09-21T02:03:59.073213+0000","last_clean":"2024-09-21T02:03:59.073213+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:03:59.073213+0000","last_undegraded":"2024-09-21T02:03:59.073213+0000","last_fullsized":"2024-09-21T02:03:59.073213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085286+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:03:58.085286+0000","last_peered":"2024-09-21T02:03:58.085286+0000","last_clean":"2024-09-21T02:03:58.085286+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:03:58.085286+0000","last_undegraded":"2024-09-21T02:03:58.085286+0000","last_fullsized":"2024-09-21T02:03:58.085286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:17.155275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075220+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:03:59.075220+0000","last_peered":"2024-09-21T02:03:59.075220+0000","last_clean":"2024-09-21T02:03:59.075220+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:03:59.075220+0000","last_undegraded":"2024-09-21T02:03:59.075220+0000","last_fullsized":"2024-09-21T02:03:59.075220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074990+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:03:59.074990+0000","last_peered":"2024-09-21T02:03:59.074990+0000","last_clean":"2024-09-21T02:03:59.074990+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:03:59.074990+0000","last_undegraded":"2024-09-21T02:03:59.074990+0000","last_fullsized":"2024-09-21T02:03:59.074990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074223+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:03:59.074223+0000","last_peered":"2024-09-21T02:03:59.074223+0000","last_clean":"2024-09-21T02:03:59.074223+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:03:59.074223+0000","last_undegraded":"2024-09-21T02:03:59.074223+0000","last_fullsized":"2024-09-21T02:03:59.074223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074095+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:03:59.074095+0000","last_peered":"2024-09-21T02:03:59.074095+0000","last_clean":"2024-09-21T02:03:59.074095+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:03:59.074095+0000","last_undegraded":"2024-09-21T02:03:59.074095+0000","last_fullsized":"2024-09-21T02:03:59.074095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835347+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:03:59.835347+0000","last_peered":"2024-09-21T02:03:59.835347+0000","last_clean":"2024-09-21T02:03:59.835347+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:03:59.835347+0000","last_undegraded":"2024-09-21T02:03:59.835347+0000","last_fullsized":"2024-09-21T02:03:59.835347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074637+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:03:59.074637+0000","last_peered":"2024-09-21T02:03:59.074637+0000","last_clean":"2024-09-21T02:03:59.074637+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:03:59.074637+0000","last_undegraded":"2024-09-21T02:03:59.074637+0000","last_fullsized":"2024-09-21T02:03:59.074637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.087515+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:03:58.087515+0000","last_peered":"2024-09-21T02:03:58.087515+0000","last_clean":"2024-09-21T02:03:58.087515+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:03:58.087515+0000","last_undegraded":"2024-09-21T02:03:58.087515+0000","last_fullsized":"2024-09-21T02:03:58.087515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:45.822704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.086382+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:03:58.086382+0000","last_peered":"2024-09-21T02:03:58.086382+0000","last_clean":"2024-09-21T02:03:58.086382+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:03:58.086382+0000","last_undegraded":"2024-09-21T02:03:58.086382+0000","last_fullsized":"2024-09-21T02:03:58.086382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:41.786759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781637+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:03:59.781637+0000","last_peered":"2024-09-21T02:03:59.781637+0000","last_clean":"2024-09-21T02:03:59.781637+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:03:59.781637+0000","last_undegraded":"2024-09-21T02:03:59.781637+0000","last_fullsized":"2024-09-21T02:03:59.781637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835918+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:03:59.835918+0000","last_peered":"2024-09-21T02:03:59.835918+0000","last_clean":"2024-09-21T02:03:59.835918+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:03:59.835918+0000","last_undegraded":"2024-09-21T02:03:59.835918+0000","last_fullsized":"2024-09-21T02:03:59.835918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.084346+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:03:58.084346+0000","last_peered":"2024-09-21T02:03:58.084346+0000","last_clean":"2024-09-21T02:03:58.084346+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:03:58.084346+0000","last_undegraded":"2024-09-21T02:03:58.084346+0000","last_fullsized":"2024-09-21T02:03:58.084346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:12.494411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072763+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:03:59.072763+0000","last_peered":"2024-09-21T02:03:59.072763+0000","last_clean":"2024-09-21T02:03:59.072763+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:03:59.072763+0000","last_undegraded":"2024-09-21T02:03:59.072763+0000","last_fullsized":"2024-09-21T02:03:59.072763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075548+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:03:59.075548+0000","last_peered":"2024-09-21T02:03:59.075548+0000","last_clean":"2024-09-21T02:03:59.075548+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:03:59.075548+0000","last_undegraded":"2024-09-21T02:03:59.075548+0000","last_fullsized":"2024-09-21T02:03:59.075548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075910+0000","last_change":"2024-09-21T02:03:58.074570+0000","last_active":"2024-09-21T02:03:59.075910+0000","last_peered":"2024-09-21T02:03:59.075910+0000","last_clean":"2024-09-21T02:03:59.075910+0000","last_became_active":"2024-09-21T02:03:58.074271+0000","last_became_peered":"2024-09-21T02:03:58.074271+0000","last_unstale":"2024-09-21T02:03:59.075910+0000","last_undegraded":"2024-09-21T02:03:59.075910+0000","last_fullsized":"2024-09-21T02:03:59.075910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.292792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835423+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:03:59.835423+0000","last_peered":"2024-09-21T02:03:59.835423+0000","last_clean":"2024-09-21T02:03:59.835423+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:03:59.835423+0000","last_undegraded":"2024-09-21T02:03:59.835423+0000","last_fullsized":"2024-09-21T02:03:59.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076132+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:03:59.076132+0000","last_peered":"2024-09-21T02:03:59.076132+0000","last_clean":"2024-09-21T02:03:59.076132+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:03:59.076132+0000","last_undegraded":"2024-09-21T02:03:59.076132+0000","last_fullsized":"2024-09-21T02:03:59.076132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.089850+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:03:58.089850+0000","last_peered":"2024-09-21T02:03:58.089850+0000","last_clean":"2024-09-21T02:03:58.089850+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:03:58.089850+0000","last_undegraded":"2024-09-21T02:03:58.089850+0000","last_fullsized":"2024-09-21T02:03:58.089850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:04.159342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085783+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:03:58.085783+0000","last_peered":"2024-09-21T02:03:58.085783+0000","last_clean":"2024-09-21T02:03:58.085783+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:03:58.085783+0000","last_undegraded":"2024-09-21T02:03:58.085783+0000","last_fullsized":"2024-09-21T02:03:58.085783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:16.364344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075251+0000","last_change":"2024-09-21T02:03:58.076378+0000","last_active":"2024-09-21T02:03:59.075251+0000","last_peered":"2024-09-21T02:03:59.075251+0000","last_clean":"2024-09-21T02:03:59.075251+0000","last_became_active":"2024-09-21T02:03:58.076075+0000","last_became_peered":"2024-09-21T02:03:58.076075+0000","last_unstale":"2024-09-21T02:03:59.075251+0000","last_undegraded":"2024-09-21T02:03:59.075251+0000","last_fullsized":"2024-09-21T02:03:59.075251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:54.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835596+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:03:59.835596+0000","last_peered":"2024-09-21T02:03:59.835596+0000","last_clean":"2024-09-21T02:03:59.835596+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:03:59.835596+0000","last_undegraded":"2024-09-21T02:03:59.835596+0000","last_fullsized":"2024-09-21T02:03:59.835596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074571+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:03:59.074571+0000","last_peered":"2024-09-21T02:03:59.074571+0000","last_clean":"2024-09-21T02:03:59.074571+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:03:59.074571+0000","last_undegraded":"2024-09-21T02:03:59.074571+0000","last_fullsized":"2024-09-21T02:03:59.074571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836762+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:03:59.836762+0000","last_peered":"2024-09-21T02:03:59.836762+0000","last_clean":"2024-09-21T02:03:59.836762+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:03:59.836762+0000","last_undegraded":"2024-09-21T02:03:59.836762+0000","last_fullsized":"2024-09-21T02:03:59.836762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085813+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:03:58.085813+0000","last_peered":"2024-09-21T02:03:58.085813+0000","last_clean":"2024-09-21T02:03:58.085813+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:03:58.085813+0000","last_undegraded":"2024-09-21T02:03:58.085813+0000","last_fullsized":"2024-09-21T02:03:58.085813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:41.739565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073053+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:03:59.073053+0000","last_peered":"2024-09-21T02:03:59.073053+0000","last_clean":"2024-09-21T02:03:59.073053+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:03:59.073053+0000","last_undegraded":"2024-09-21T02:03:59.073053+0000","last_fullsized":"2024-09-21T02:03:59.073053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836383+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:03:59.836383+0000","last_peered":"2024-09-21T02:03:59.836383+0000","last_clean":"2024-09-21T02:03:59.836383+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:03:59.836383+0000","last_undegraded":"2024-09-21T02:03:59.836383+0000","last_fullsized":"2024-09-21T02:03:59.836383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085757+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:03:58.085757+0000","last_peered":"2024-09-21T02:03:58.085757+0000","last_clean":"2024-09-21T02:03:58.085757+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:03:58.085757+0000","last_undegraded":"2024-09-21T02:03:58.085757+0000","last_fullsized":"2024-09-21T02:03:58.085757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:43.473957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075554+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:03:59.075554+0000","last_peered":"2024-09-21T02:03:59.075554+0000","last_clean":"2024-09-21T02:03:59.075554+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:03:59.075554+0000","last_undegraded":"2024-09-21T02:03:59.075554+0000","last_fullsized":"2024-09-21T02:03:59.075554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835391+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:03:59.835391+0000","last_peered":"2024-09-21T02:03:59.835391+0000","last_clean":"2024-09-21T02:03:59.835391+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:03:59.835391+0000","last_undegraded":"2024-09-21T02:03:59.835391+0000","last_fullsized":"2024-09-21T02:03:59.835391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.087474+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:03:58.087474+0000","last_peered":"2024-09-21T02:03:58.087474+0000","last_clean":"2024-09-21T02:03:58.087474+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:03:58.087474+0000","last_undegraded":"2024-09-21T02:03:58.087474+0000","last_fullsized":"2024-09-21T02:03:58.087474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:11.729550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781525+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:03:59.781525+0000","last_peered":"2024-09-21T02:03:59.781525+0000","last_clean":"2024-09-21T02:03:59.781525+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:03:59.781525+0000","last_undegraded":"2024-09-21T02:03:59.781525+0000","last_fullsized":"2024-09-21T02:03:59.781525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836683+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:03:59.836683+0000","last_peered":"2024-09-21T02:03:59.836683+0000","last_clean":"2024-09-21T02:03:59.836683+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:03:59.836683+0000","last_undegraded":"2024-09-21T02:03:59.836683+0000","last_fullsized":"2024-09-21T02:03:59.836683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085712+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:03:58.085712+0000","last_peered":"2024-09-21T02:03:58.085712+0000","last_clean":"2024-09-21T02:03:58.085712+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:03:58.085712+0000","last_undegraded":"2024-09-21T02:03:58.085712+0000","last_fullsized":"2024-09-21T02:03:58.085712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:46.972262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.067310+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:03:58.067310+0000","last_peered":"2024-09-21T02:03:58.067310+0000","last_clean":"2024-09-21T02:03:58.067310+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:03:58.067310+0000","last_undegraded":"2024-09-21T02:03:58.067310+0000","last_fullsized":"2024-09-21T02:03:58.067310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835432+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:03:59.835432+0000","last_peered":"2024-09-21T02:03:59.835432+0000","last_clean":"2024-09-21T02:03:59.835432+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:03:59.835432+0000","last_undegraded":"2024-09-21T02:03:59.835432+0000","last_fullsized":"2024-09-21T02:03:59.835432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835573+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:03:59.835573+0000","last_peered":"2024-09-21T02:03:59.835573+0000","last_clean":"2024-09-21T02:03:59.835573+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:03:59.835573+0000","last_undegraded":"2024-09-21T02:03:59.835573+0000","last_fullsized":"2024-09-21T02:03:59.835573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074657+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:03:59.074657+0000","last_peered":"2024-09-21T02:03:59.074657+0000","last_clean":"2024-09-21T02:03:59.074657+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:03:59.074657+0000","last_undegraded":"2024-09-21T02:03:59.074657+0000","last_fullsized":"2024-09-21T02:03:59.074657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072701+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:03:59.072701+0000","last_peered":"2024-09-21T02:03:59.072701+0000","last_clean":"2024-09-21T02:03:59.072701+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:03:59.072701+0000","last_undegraded":"2024-09-21T02:03:59.072701+0000","last_fullsized":"2024-09-21T02:03:59.072701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.066693+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:03:58.066693+0000","last_peered":"2024-09-21T02:03:58.066693+0000","last_clean":"2024-09-21T02:03:58.066693+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:03:58.066693+0000","last_undegraded":"2024-09-21T02:03:58.066693+0000","last_fullsized":"2024-09-21T02:03:58.066693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074748+0000","last_change":"2024-09-21T02:03:58.075434+0000","last_active":"2024-09-21T02:03:59.074748+0000","last_peered":"2024-09-21T02:03:59.074748+0000","last_clean":"2024-09-21T02:03:59.074748+0000","last_became_active":"2024-09-21T02:03:58.075130+0000","last_became_peered":"2024-09-21T02:03:58.075130+0000","last_unstale":"2024-09-21T02:03:59.074748+0000","last_undegraded":"2024-09-21T02:03:59.074748+0000","last_fullsized":"2024-09-21T02:03:59.074748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:56.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074818+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:03:59.074818+0000","last_peered":"2024-09-21T02:03:59.074818+0000","last_clean":"2024-09-21T02:03:59.074818+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:03:59.074818+0000","last_undegraded":"2024-09-21T02:03:59.074818+0000","last_fullsized":"2024-09-21T02:03:59.074818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085666+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:03:58.085666+0000","last_peered":"2024-09-21T02:03:58.085666+0000","last_clean":"2024-09-21T02:03:58.085666+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:03:58.085666+0000","last_undegraded":"2024-09-21T02:03:58.085666+0000","last_fullsized":"2024-09-21T02:03:58.085666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:30.459067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072952+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:03:59.072952+0000","last_peered":"2024-09-21T02:03:59.072952+0000","last_clean":"2024-09-21T02:03:59.072952+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:03:59.072952+0000","last_undegraded":"2024-09-21T02:03:59.072952+0000","last_fullsized":"2024-09-21T02:03:59.072952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781695+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:03:59.781695+0000","last_peered":"2024-09-21T02:03:59.781695+0000","last_clean":"2024-09-21T02:03:59.781695+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:03:59.781695+0000","last_undegraded":"2024-09-21T02:03:59.781695+0000","last_fullsized":"2024-09-21T02:03:59.781695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.067848+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:03:58.067848+0000","last_peered":"2024-09-21T02:03:58.067848+0000","last_clean":"2024-09-21T02:03:58.067848+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:03:58.067848+0000","last_undegraded":"2024-09-21T02:03:58.067848+0000","last_fullsized":"2024-09-21T02:03:58.067848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075568+0000","last_change":"2024-09-21T02:03:58.080947+0000","last_active":"2024-09-21T02:03:59.075568+0000","last_peered":"2024-09-21T02:03:59.075568+0000","last_clean":"2024-09-21T02:03:59.075568+0000","last_became_active":"2024-09-21T02:03:58.080632+0000","last_became_peered":"2024-09-21T02:03:58.080632+0000","last_unstale":"2024-09-21T02:03:59.075568+0000","last_undegraded":"2024-09-21T02:03:59.075568+0000","last_fullsized":"2024-09-21T02:03:59.075568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:40.307689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.068339+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:03:58.068339+0000","last_peered":"2024-09-21T02:03:58.068339+0000","last_clean":"2024-09-21T02:03:58.068339+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:03:58.068339+0000","last_undegraded":"2024-09-21T02:03:58.068339+0000","last_fullsized":"2024-09-21T02:03:58.068339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085448+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:03:58.085448+0000","last_peered":"2024-09-21T02:03:58.085448+0000","last_clean":"2024-09-21T02:03:58.085448+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:03:58.085448+0000","last_undegraded":"2024-09-21T02:03:58.085448+0000","last_fullsized":"2024-09-21T02:03:58.085448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:22.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075336+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:03:59.075336+0000","last_peered":"2024-09-21T02:03:59.075336+0000","last_clean":"2024-09-21T02:03:59.075336+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:03:59.075336+0000","last_undegraded":"2024-09-21T02:03:59.075336+0000","last_fullsized":"2024-09-21T02:03:59.075336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075210+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:03:59.075210+0000","last_peered":"2024-09-21T02:03:59.075210+0000","last_clean":"2024-09-21T02:03:59.075210+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:03:59.075210+0000","last_undegraded":"2024-09-21T02:03:59.075210+0000","last_fullsized":"2024-09-21T02:03:59.075210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.066464+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:03:58.066464+0000","last_peered":"2024-09-21T02:03:58.066464+0000","last_clean":"2024-09-21T02:03:58.066464+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:03:58.066464+0000","last_undegraded":"2024-09-21T02:03:58.066464+0000","last_fullsized":"2024-09-21T02:03:58.066464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.086424+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:03:58.086424+0000","last_peered":"2024-09-21T02:03:58.086424+0000","last_clean":"2024-09-21T02:03:58.086424+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:03:58.086424+0000","last_undegraded":"2024-09-21T02:03:58.086424+0000","last_fullsized":"2024-09-21T02:03:58.086424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:08:10.961713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835098+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:03:59.835098+0000","last_peered":"2024-09-21T02:03:59.835098+0000","last_clean":"2024-09-21T02:03:59.835098+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:03:59.835098+0000","last_undegraded":"2024-09-21T02:03:59.835098+0000","last_fullsized":"2024-09-21T02:03:59.835098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074582+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:03:59.074582+0000","last_peered":"2024-09-21T02:03:59.074582+0000","last_clean":"2024-09-21T02:03:59.074582+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:03:59.074582+0000","last_undegraded":"2024-09-21T02:03:59.074582+0000","last_fullsized":"2024-09-21T02:03:59.074582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781506+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:03:59.781506+0000","last_peered":"2024-09-21T02:03:59.781506+0000","last_clean":"2024-09-21T02:03:59.781506+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:03:59.781506+0000","last_undegraded":"2024-09-21T02:03:59.781506+0000","last_fullsized":"2024-09-21T02:03:59.781506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074555+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:03:59.074555+0000","last_peered":"2024-09-21T02:03:59.074555+0000","last_clean":"2024-09-21T02:03:59.074555+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:03:59.074555+0000","last_undegraded":"2024-09-21T02:03:59.074555+0000","last_fullsized":"2024-09-21T02:03:59.074555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835494+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:03:59.835494+0000","last_peered":"2024-09-21T02:03:59.835494+0000","last_clean":"2024-09-21T02:03:59.835494+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:03:59.835494+0000","last_undegraded":"2024-09-21T02:03:59.835494+0000","last_fullsized":"2024-09-21T02:03:59.835494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074911+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:03:59.074911+0000","last_peered":"2024-09-21T02:03:59.074911+0000","last_clean":"2024-09-21T02:03:59.074911+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:03:59.074911+0000","last_undegraded":"2024-09-21T02:03:59.074911+0000","last_fullsized":"2024-09-21T02:03:59.074911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073063+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:03:59.073063+0000","last_peered":"2024-09-21T02:03:59.073063+0000","last_clean":"2024-09-21T02:03:59.073063+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:03:59.073063+0000","last_undegraded":"2024-09-21T02:03:59.073063+0000","last_fullsized":"2024-09-21T02:03:59.073063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075681+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:03:59.075681+0000","last_peered":"2024-09-21T02:03:59.075681+0000","last_clean":"2024-09-21T02:03:59.075681+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:03:59.075681+0000","last_undegraded":"2024-09-21T02:03:59.075681+0000","last_fullsized":"2024-09-21T02:03:59.075681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835624+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:03:59.835624+0000","last_peered":"2024-09-21T02:03:59.835624+0000","last_clean":"2024-09-21T02:03:59.835624+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:03:59.835624+0000","last_undegraded":"2024-09-21T02:03:59.835624+0000","last_fullsized":"2024-09-21T02:03:59.835624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075817+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:03:59.075817+0000","last_peered":"2024-09-21T02:03:59.075817+0000","last_clean":"2024-09-21T02:03:59.075817+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:03:59.075817+0000","last_undegraded":"2024-09-21T02:03:59.075817+0000","last_fullsized":"2024-09-21T02:03:59.075817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074041+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:03:59.074041+0000","last_peered":"2024-09-21T02:03:59.074041+0000","last_clean":"2024-09-21T02:03:59.074041+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:03:59.074041+0000","last_undegraded":"2024-09-21T02:03:59.074041+0000","last_fullsized":"2024-09-21T02:03:59.074041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074376+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:03:59.074376+0000","last_peered":"2024-09-21T02:03:59.074376+0000","last_clean":"2024-09-21T02:03:59.074376+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:03:59.074376+0000","last_undegraded":"2024-09-21T02:03:59.074376+0000","last_fullsized":"2024-09-21T02:03:59.074376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835923+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:03:59.835923+0000","last_peered":"2024-09-21T02:03:59.835923+0000","last_clean":"2024-09-21T02:03:59.835923+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:03:59.835923+0000","last_undegraded":"2024-09-21T02:03:59.835923+0000","last_fullsized":"2024-09-21T02:03:59.835923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073892+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:03:59.073892+0000","last_peered":"2024-09-21T02:03:59.073892+0000","last_clean":"2024-09-21T02:03:59.073892+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:03:59.073892+0000","last_undegraded":"2024-09-21T02:03:59.073892+0000","last_fullsized":"2024-09-21T02:03:59.073892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085608+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:03:58.085608+0000","last_peered":"2024-09-21T02:03:58.085608+0000","last_clean":"2024-09-21T02:03:58.085608+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:03:58.085608+0000","last_undegraded":"2024-09-21T02:03:58.085608+0000","last_fullsized":"2024-09-21T02:03:58.085608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:40.368766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835266+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:03:59.835266+0000","last_peered":"2024-09-21T02:03:59.835266+0000","last_clean":"2024-09-21T02:03:59.835266+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:03:59.835266+0000","last_undegraded":"2024-09-21T02:03:59.835266+0000","last_fullsized":"2024-09-21T02:03:59.835266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072781+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:03:59.072781+0000","last_peered":"2024-09-21T02:03:59.072781+0000","last_clean":"2024-09-21T02:03:59.072781+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:03:59.072781+0000","last_undegraded":"2024-09-21T02:03:59.072781+0000","last_fullsized":"2024-09-21T02:03:59.072781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075493+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:03:59.075493+0000","last_peered":"2024-09-21T02:03:59.075493+0000","last_clean":"2024-09-21T02:03:59.075493+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:03:59.075493+0000","last_undegraded":"2024-09-21T02:03:59.075493+0000","last_fullsized":"2024-09-21T02:03:59.075493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781505+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:03:59.781505+0000","last_peered":"2024-09-21T02:03:59.781505+0000","last_clean":"2024-09-21T02:03:59.781505+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:03:59.781505+0000","last_undegraded":"2024-09-21T02:03:59.781505+0000","last_fullsized":"2024-09-21T02:03:59.781505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072896+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:03:59.072896+0000","last_peered":"2024-09-21T02:03:59.072896+0000","last_clean":"2024-09-21T02:03:59.072896+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:03:59.072896+0000","last_undegraded":"2024-09-21T02:03:59.072896+0000","last_fullsized":"2024-09-21T02:03:59.072896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076487+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:03:59.076487+0000","last_peered":"2024-09-21T02:03:59.076487+0000","last_clean":"2024-09-21T02:03:59.076487+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:03:59.076487+0000","last_undegraded":"2024-09-21T02:03:59.076487+0000","last_fullsized":"2024-09-21T02:03:59.076487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075934+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:03:59.075934+0000","last_peered":"2024-09-21T02:03:59.075934+0000","last_clean":"2024-09-21T02:03:59.075934+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:03:59.075934+0000","last_undegraded":"2024-09-21T02:03:59.075934+0000","last_fullsized":"2024-09-21T02:03:59.075934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073021+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:03:59.073021+0000","last_peered":"2024-09-21T02:03:59.073021+0000","last_clean":"2024-09-21T02:03:59.073021+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:03:59.073021+0000","last_undegraded":"2024-09-21T02:03:59.073021+0000","last_fullsized":"2024-09-21T02:03:59.073021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073068+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:03:59.073068+0000","last_peered":"2024-09-21T02:03:59.073068+0000","last_clean":"2024-09-21T02:03:59.073068+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:03:59.073068+0000","last_undegraded":"2024-09-21T02:03:59.073068+0000","last_fullsized":"2024-09-21T02:03:59.073068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836949+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:03:59.836949+0000","last_peered":"2024-09-21T02:03:59.836949+0000","last_clean":"2024-09-21T02:03:59.836949+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:03:59.836949+0000","last_undegraded":"2024-09-21T02:03:59.836949+0000","last_fullsized":"2024-09-21T02:03:59.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.066772+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:03:58.066772+0000","last_peered":"2024-09-21T02:03:58.066772+0000","last_clean":"2024-09-21T02:03:58.066772+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:03:58.066772+0000","last_undegraded":"2024-09-21T02:03:58.066772+0000","last_fullsized":"2024-09-21T02:03:58.066772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085374+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:03:58.085374+0000","last_peered":"2024-09-21T02:03:58.085374+0000","last_clean":"2024-09-21T02:03:58.085374+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:03:58.085374+0000","last_undegraded":"2024-09-21T02:03:58.085374+0000","last_fullsized":"2024-09-21T02:03:58.085374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:46.637904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073699+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:03:59.073699+0000","last_peered":"2024-09-21T02:03:59.073699+0000","last_clean":"2024-09-21T02:03:59.073699+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:03:59.073699+0000","last_undegraded":"2024-09-21T02:03:59.073699+0000","last_fullsized":"2024-09-21T02:03:59.073699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781583+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:03:59.781583+0000","last_peered":"2024-09-21T02:03:59.781583+0000","last_clean":"2024-09-21T02:03:59.781583+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:03:59.781583+0000","last_undegraded":"2024-09-21T02:03:59.781583+0000","last_fullsized":"2024-09-21T02:03:59.781583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072755+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:03:59.072755+0000","last_peered":"2024-09-21T02:03:59.072755+0000","last_clean":"2024-09-21T02:03:59.072755+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:03:59.072755+0000","last_undegraded":"2024-09-21T02:03:59.072755+0000","last_fullsized":"2024-09-21T02:03:59.072755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835669+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:03:59.835669+0000","last_peered":"2024-09-21T02:03:59.835669+0000","last_clean":"2024-09-21T02:03:59.835669+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:03:59.835669+0000","last_undegraded":"2024-09-21T02:03:59.835669+0000","last_fullsized":"2024-09-21T02:03:59.835669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072863+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:03:59.072863+0000","last_peered":"2024-09-21T02:03:59.072863+0000","last_clean":"2024-09-21T02:03:59.072863+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:03:59.072863+0000","last_undegraded":"2024-09-21T02:03:59.072863+0000","last_fullsized":"2024-09-21T02:03:59.072863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836615+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:03:59.836615+0000","last_peered":"2024-09-21T02:03:59.836615+0000","last_clean":"2024-09-21T02:03:59.836615+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:03:59.836615+0000","last_undegraded":"2024-09-21T02:03:59.836615+0000","last_fullsized":"2024-09-21T02:03:59.836615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075290+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:03:59.075290+0000","last_peered":"2024-09-21T02:03:59.075290+0000","last_clean":"2024-09-21T02:03:59.075290+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:03:59.075290+0000","last_undegraded":"2024-09-21T02:03:59.075290+0000","last_fullsized":"2024-09-21T02:03:59.075290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.084565+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:03:58.084565+0000","last_peered":"2024-09-21T02:03:58.084565+0000","last_clean":"2024-09-21T02:03:58.084565+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:03:58.084565+0000","last_undegraded":"2024-09-21T02:03:58.084565+0000","last_fullsized":"2024-09-21T02:03:58.084565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:08.508247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076421+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:03:59.076421+0000","last_peered":"2024-09-21T02:03:59.076421+0000","last_clean":"2024-09-21T02:03:59.076421+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:03:59.076421+0000","last_undegraded":"2024-09-21T02:03:59.076421+0000","last_fullsized":"2024-09-21T02:03:59.076421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074280+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:03:59.074280+0000","last_peered":"2024-09-21T02:03:59.074280+0000","last_clean":"2024-09-21T02:03:59.074280+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:03:59.074280+0000","last_undegraded":"2024-09-21T02:03:59.074280+0000","last_fullsized":"2024-09-21T02:03:59.074280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075820+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:03:59.075820+0000","last_peered":"2024-09-21T02:03:59.075820+0000","last_clean":"2024-09-21T02:03:59.075820+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:03:59.075820+0000","last_undegraded":"2024-09-21T02:03:59.075820+0000","last_fullsized":"2024-09-21T02:03:59.075820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075758+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:03:59.075758+0000","last_peered":"2024-09-21T02:03:59.075758+0000","last_clean":"2024-09-21T02:03:59.075758+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:03:59.075758+0000","last_undegraded":"2024-09-21T02:03:59.075758+0000","last_fullsized":"2024-09-21T02:03:59.075758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.085611+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:03:58.085611+0000","last_peered":"2024-09-21T02:03:58.085611+0000","last_clean":"2024-09-21T02:03:58.085611+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:03:58.085611+0000","last_undegraded":"2024-09-21T02:03:58.085611+0000","last_fullsized":"2024-09-21T02:03:58.085611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:56.367906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837047+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:03:59.837047+0000","last_peered":"2024-09-21T02:03:59.837047+0000","last_clean":"2024-09-21T02:03:59.837047+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:03:59.837047+0000","last_undegraded":"2024-09-21T02:03:59.837047+0000","last_fullsized":"2024-09-21T02:03:59.837047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837178+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:03:59.837178+0000","last_peered":"2024-09-21T02:03:59.837178+0000","last_clean":"2024-09-21T02:03:59.837178+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:03:59.837178+0000","last_undegraded":"2024-09-21T02:03:59.837178+0000","last_fullsized":"2024-09-21T02:03:59.837178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:03:58.067826+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:03:58.067826+0000","last_peered":"2024-09-21T02:03:58.067826+0000","last_clean":"2024-09-21T02:03:58.067826+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:03:58.067826+0000","last_undegraded":"2024-09-21T02:03:58.067826+0000","last_fullsized":"2024-09-21T02:03:58.067826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":802816,"data_stored":663115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35548,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336292,"statfs":{"total":96636764160,"available":96600363008,"internally_reserved":0,"allocated":196608,"data_stored":66447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":790528,"data_stored":656815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:03.442 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:04:03.443 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T02:04:03.443 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T02:04:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.591+0000 7f5ea3c30640 1 -- 172.21.15.88:0/3183894747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 msgr2=0x7f5e88005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.591+0000 7f5ea3c30640 1 --2- 172.21.15.88:0/3183894747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 0x7f5e88005150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c071050 tx=0x7f5e88005830 comp rx=0 tx=0).stop 2024-09-21T02:04:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.591+0000 7f5ea21cd640 1 -- 172.21.15.88:0/3183894747 reap_dead start 2024-09-21T02:04:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.591+0000 7f5ea3c30640 1 -- 172.21.15.88:0/3183894747 shutdown_connections 2024-09-21T02:04:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.591+0000 7f5ea3c30640 1 -- 172.21.15.88:0/3183894747 >> 172.21.15.88:0/3183894747 conn(0x7f5e9c072f20 msgr2=0x7f5e9c073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.591+0000 7f5ea3c30640 1 -- 172.21.15.88:0/3183894747 shutdown_connections 2024-09-21T02:04:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.591+0000 7f5ea3c30640 1 -- 172.21.15.88:0/3183894747 wait complete. 2024-09-21T02:04:03.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.593+0000 7f5ea3c30640 1 Processor -- start 2024-09-21T02:04:03.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.593+0000 7f5ea3c30640 1 -- start start 2024-09-21T02:04:03.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea3c30640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 0x7f5e9c1ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea3c30640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e9c154f00 0x7f5e9c149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea3c30640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e9c146910 0x7f5e9c146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea3c30640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c0c5ae0 con 0x7f5e88004d40 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea3c30640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c072230 con 0x7f5e9c146910 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea3c30640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c153d00 con 0x7f5e9c154f00 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 0x7f5e9c1ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 0x7f5e9c1ef750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52588/0 (socket says 172.21.15.88:52588) 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 -- 172.21.15.88:0/2404295885 learned_addr learned my addr 172.21.15.88:0/2404295885 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea21cd640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e9c146910 0x7f5e9c146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea11cb640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e9c154f00 0x7f5e9c149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 -- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e9c154f00 msgr2=0x7f5e9c149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e9c154f00 0x7f5e9c149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 -- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e9c146910 msgr2=0x7f5e9c146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e9c146910 0x7f5e9c146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea19cc640 1 -- 172.21.15.88:0/2404295885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c1efc90 con 0x7f5e88004d40 2024-09-21T02:04:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.594+0000 7f5ea21cd640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e9c146910 0x7f5e9c146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.595+0000 7f5ea19cc640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 0x7f5e9c1ef750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e98011ae0 tx=0x7f5e98009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.595+0000 7f5e927fc640 1 -- 172.21.15.88:0/2404295885 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9801e070 con 0x7f5e88004d40 2024-09-21T02:04:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.595+0000 7f5e927fc640 1 -- 172.21.15.88:0/2404295885 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e98004460 con 0x7f5e88004d40 2024-09-21T02:04:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.595+0000 7f5e927fc640 1 -- 172.21.15.88:0/2404295885 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800fc60 con 0x7f5e88004d40 2024-09-21T02:04:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.595+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9c147260 con 0x7f5e88004d40 2024-09-21T02:04:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.595+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9c14ad90 con 0x7f5e88004d40 2024-09-21T02:04:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.596+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e9c1efe60 con 0x7f5e88004d40 2024-09-21T02:04:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.597+0000 7f5e927fc640 1 -- 172.21.15.88:0/2404295885 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5e98021070 con 0x7f5e88004d40 2024-09-21T02:04:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.597+0000 7f5e927fc640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5e80078bc0 0x7f5e8007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.597+0000 7f5e927fc640 1 -- 172.21.15.88:0/2404295885 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5e9809b6f0 con 0x7f5e88004d40 2024-09-21T02:04:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.597+0000 7f5ea11cb640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5e80078bc0 0x7f5e8007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.598+0000 7f5ea11cb640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5e80078bc0 0x7f5e8007b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c1e9d30 tx=0x7f5e8c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.601+0000 7f5e927fc640 1 -- 172.21.15.88:0/2404295885 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e98063630 con 0x7f5e88004d40 2024-09-21T02:04:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.778+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f5e9c14fcf0 con 0x7f5e88004d40 2024-09-21T02:04:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.778+0000 7f5e927fc640 1 -- 172.21.15.88:0/2404295885 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v36) ==== 86+0+160 (secure 0 0 0) 0x7f5e98068060 con 0x7f5e88004d40 2024-09-21T02:04:03.778 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:03.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5e80078bc0 msgr2=0x7f5e8007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5e80078bc0 0x7f5e8007b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c1e9d30 tx=0x7f5e8c005f90 comp rx=0 tx=0).stop 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 msgr2=0x7f5e9c1ef750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 0x7f5e9c1ef750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5e98011ae0 tx=0x7f5e98009e90 comp rx=0 tx=0).stop 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 shutdown_connections 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5e80078bc0 0x7f5e8007b0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e9c146910 0x7f5e9c146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e9c154f00 0x7f5e9c149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 --2- 172.21.15.88:0/2404295885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e88004d40 0x7f5e9c1ef750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 >> 172.21.15.88:0/2404295885 conn(0x7f5e9c072f20 msgr2=0x7f5e9c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 shutdown_connections 2024-09-21T02:04:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.780+0000 7f5ea3c30640 1 -- 172.21.15.88:0/2404295885 wait complete. 2024-09-21T02:04:03.791 INFO:teuthology.orchestra.run.smithi088.stdout:[{"version":36,"timestamp":"2024-09-21T02:03:45.310678+0000","name":"","changes":[{"name":"osd.3/osd_mclock_max_capacity_iops_ssd","new_value":"4885.371261"}]}] 2024-09-21T02:04:03.791 INFO:tasks.ceph_manager:config epoch is 36 2024-09-21T02:04:03.791 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T02:04:03.792 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T02:04:03.792 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T02:04:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 -- 172.21.15.88:0/2880016395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 msgr2=0x7f39080cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 --2- 172.21.15.88:0/2880016395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 0x7f39080cbbd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3900005fd0 tx=0x7f390002a8d0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 -- 172.21.15.88:0/2880016395 shutdown_connections 2024-09-21T02:04:03.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 --2- 172.21.15.88:0/2880016395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 0x7f39080cbbd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 --2- 172.21.15.88:0/2880016395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390815f970 0x7f390806cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 -- 172.21.15.88:0/2880016395 >> 172.21.15.88:0/2880016395 conn(0x7f39080cd9f0 msgr2=0x7f39080cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:03.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 -- 172.21.15.88:0/2880016395 shutdown_connections 2024-09-21T02:04:03.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.904+0000 7f39102b4640 1 -- 172.21.15.88:0/2880016395 wait complete. 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.906+0000 7f39102b4640 1 Processor -- start 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.906+0000 7f39102b4640 1 -- start start 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f39102b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 0x7f39081e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f39102b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39080ccd00 0x7f3908152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f39102b4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390815f970 0x7f3908152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f39102b4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f390815fd80 con 0x7f390815f970 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f39102b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f390815c640 con 0x7f390806d2a0 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f39102b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f390806dcd0 con 0x7f39080ccd00 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390815f970 0x7f3908152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390815f970 0x7f3908152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52592/0 (socket says 172.21.15.88:52592) 2024-09-21T02:04:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 -- 172.21.15.88:0/1678721960 learned_addr learned my addr 172.21.15.88:0/1678721960 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e050640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 0x7f39081e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390d84f640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39080ccd00 0x7f3908152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 -- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39080ccd00 msgr2=0x7f3908152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39080ccd00 0x7f3908152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 -- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 msgr2=0x7f39081e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 0x7f39081e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e851640 1 -- 172.21.15.88:0/1678721960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39081e5d60 con 0x7f390815f970 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.907+0000 7f390e050640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 0x7f39081e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.908+0000 7f390e851640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390815f970 0x7f3908152990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3900009cd0 tx=0x7f3900032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.908+0000 7f38feffd640 1 -- 172.21.15.88:0/1678721960 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900037040 con 0x7f390815f970 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.908+0000 7f38feffd640 1 -- 172.21.15.88:0/1678721960 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3900003c60 con 0x7f390815f970 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.908+0000 7f38feffd640 1 -- 172.21.15.88:0/1678721960 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390003c990 con 0x7f390815f970 2024-09-21T02:04:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.908+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3908152ed0 con 0x7f390815f970 2024-09-21T02:04:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.908+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081e4d40 con 0x7f390815f970 2024-09-21T02:04:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.909+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390806cd60 con 0x7f390815f970 2024-09-21T02:04:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.909+0000 7f38feffd640 1 -- 172.21.15.88:0/1678721960 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f390003a070 con 0x7f390815f970 2024-09-21T02:04:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.910+0000 7f38feffd640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f38ec078b90 0x7f38ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.910+0000 7f38feffd640 1 -- 172.21.15.88:0/1678721960 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f39000bba80 con 0x7f390815f970 2024-09-21T02:04:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.910+0000 7f390e050640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f38ec078b90 0x7f38ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.910+0000 7f390e050640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f38ec078b90 0x7f38ec07b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3904001eb0 tx=0x7f3904009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:03.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:03.914+0000 7f38feffd640 1 -- 172.21.15.88:0/1678721960 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3900041020 con 0x7f390815f970 2024-09-21T02:04:04.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.138+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f39080cb040 con 0x7f390815f970 2024-09-21T02:04:04.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.141+0000 7f38feffd640 1 -- 172.21.15.88:0/1678721960 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194721 (secure 0 0 0) 0x7f3900083fe0 con 0x7f390815f970 2024-09-21T02:04:04.144 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f38ec078b90 msgr2=0x7f38ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f38ec078b90 0x7f38ec07b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3904001eb0 tx=0x7f3904009040 comp rx=0 tx=0).stop 2024-09-21T02:04:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390815f970 msgr2=0x7f3908152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390815f970 0x7f3908152990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3900009cd0 tx=0x7f3900032f30 comp rx=0 tx=0).stop 2024-09-21T02:04:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 shutdown_connections 2024-09-21T02:04:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f38ec078b90 0x7f38ec07b0a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390815f970 0x7f3908152990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39080ccd00 0x7f3908152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 --2- 172.21.15.88:0/1678721960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390806d2a0 0x7f39081e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 >> 172.21.15.88:0/1678721960 conn(0x7f39080cd9f0 msgr2=0x7f3908155560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:04.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 shutdown_connections 2024-09-21T02:04:04.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.144+0000 7f39102b4640 1 -- 172.21.15.88:0/1678721960 wait complete. 2024-09-21T02:04:04.155 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":5,"flags":0,"active_gid":4117,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4016634266},{"type":"v1","addr":"172.21.15.195:6801","nonce":4016634266}]},"active_addr":"172.21.15.195:6801/4016634266","active_change":"2024-09-21T02:03:36.809804+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4109,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2355021642}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":1910922531}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":514080132}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":3076413593}]}]} 2024-09-21T02:04:04.158 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T02:04:04.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T02:04:04.158 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:04:04.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.270+0000 7fa20b815640 1 -- 172.21.15.88:0/4136713366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f0004800 msgr2=0x7fa1f0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.270+0000 7fa20b815640 1 --2- 172.21.15.88:0/4136713366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f0004800 0x7fa1f0004c10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4001050 tx=0x7fa1f0005830 comp rx=0 tx=0).stop 2024-09-21T02:04:04.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.270+0000 7fa20b815640 1 -- 172.21.15.88:0/4136713366 shutdown_connections 2024-09-21T02:04:04.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.270+0000 7fa20b815640 1 --2- 172.21.15.88:0/4136713366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f0004800 0x7fa1f0004c10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.270+0000 7fa20b815640 1 -- 172.21.15.88:0/4136713366 >> 172.21.15.88:0/4136713366 conn(0x7fa2040729d0 msgr2=0x7fa204072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:04.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.270+0000 7fa20b815640 1 -- 172.21.15.88:0/4136713366 shutdown_connections 2024-09-21T02:04:04.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.270+0000 7fa20b815640 1 -- 172.21.15.88:0/4136713366 wait complete. 2024-09-21T02:04:04.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.272+0000 7fa20b815640 1 Processor -- start 2024-09-21T02:04:04.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.272+0000 7fa20b815640 1 -- start start 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa20b815640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1f0004800 0x7fa204155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa20b815640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa204149b50 0x7fa2041444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa20b815640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2041546a0 0x7fa204154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa20b815640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa20415a5b0 con 0x7fa204149b50 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa20b815640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa204070ab0 con 0x7fa1f0004800 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa20b815640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa2040c8f30 con 0x7fa2041546a0 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa208db0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa204149b50 0x7fa2041444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa208db0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa204149b50 0x7fa2041444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52604/0 (socket says 172.21.15.88:52604) 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa208db0640 1 -- 172.21.15.88:0/3554612124 learned_addr learned my addr 172.21.15.88:0/3554612124 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.273+0000 7fa2095b1640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1f0004800 0x7fa204155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa209db2640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2041546a0 0x7fa204154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa208db0640 1 -- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2041546a0 msgr2=0x7fa204154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa208db0640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2041546a0 0x7fa204154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa208db0640 1 -- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1f0004800 msgr2=0x7fa204155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa208db0640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1f0004800 0x7fa204155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa208db0640 1 -- 172.21.15.88:0/3554612124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20414d510 con 0x7fa204149b50 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa2095b1640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1f0004800 0x7fa204155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa208db0640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa204149b50 0x7fa2041444c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1f402b0a0 tx=0x7fa1f4006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa1f9ffb640 1 -- 172.21.15.88:0/3554612124 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4003a00 con 0x7fa204149b50 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa1f9ffb640 1 -- 172.21.15.88:0/3554612124 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1f4003c90 con 0x7fa204149b50 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2041f0740 con 0x7fa204149b50 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa1f9ffb640 1 -- 172.21.15.88:0/3554612124 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f402e6e0 con 0x7fa204149b50 2024-09-21T02:04:04.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.274+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa204144a00 con 0x7fa204149b50 2024-09-21T02:04:04.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.275+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2041494b0 con 0x7fa204149b50 2024-09-21T02:04:04.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.276+0000 7fa1f9ffb640 1 -- 172.21.15.88:0/3554612124 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa1f4035020 con 0x7fa204149b50 2024-09-21T02:04:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.277+0000 7fa1f9ffb640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1e8078b90 0x7fa1e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.277+0000 7fa1f9ffb640 1 -- 172.21.15.88:0/3554612124 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f40bb0a0 con 0x7fa204149b50 2024-09-21T02:04:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.277+0000 7fa2095b1640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1e8078b90 0x7fa1e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.278+0000 7fa2095b1640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1e8078b90 0x7fa1e807b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa20406be30 tx=0x7fa200006ca0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:04.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.281+0000 7fa1f9ffb640 1 -- 172.21.15.88:0/3554612124 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1f4047030 con 0x7fa204149b50 2024-09-21T02:04:04.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.460+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa204156df0 con 0x7fa204149b50 2024-09-21T02:04:04.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.461+0000 7fa1f9ffb640 1 -- 172.21.15.88:0/3554612124 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fa1f403e070 con 0x7fa204149b50 2024-09-21T02:04:04.461 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:04.462 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":21,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:59.066086+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:04.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1e8078b90 msgr2=0x7fa1e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1e8078b90 0x7fa1e807b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa20406be30 tx=0x7fa200006ca0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa204149b50 msgr2=0x7fa2041444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa204149b50 0x7fa2041444c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1f402b0a0 tx=0x7fa1f4006d00 comp rx=0 tx=0).stop 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 shutdown_connections 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1e8078b90 0x7fa1e807b0a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa2041546a0 0x7fa204154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa204149b50 0x7fa2041444c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 --2- 172.21.15.88:0/3554612124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1f0004800 0x7fa204155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 >> 172.21.15.88:0/3554612124 conn(0x7fa2040729d0 msgr2=0x7fa2040caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 shutdown_connections 2024-09-21T02:04:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.463+0000 7fa20b815640 1 -- 172.21.15.88:0/3554612124 wait complete. 2024-09-21T02:04:04.475 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T02:04:04.475 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:04:04.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 -- 172.21.15.88:0/2808052016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0cb170 msgr2=0x7f62bc0cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 --2- 172.21.15.88:0/2808052016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0cb170 0x7f62bc0cbbd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0754d0 tx=0x7f62b402a5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 -- 172.21.15.88:0/2808052016 shutdown_connections 2024-09-21T02:04:04.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 --2- 172.21.15.88:0/2808052016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0cb170 0x7f62bc0cbbd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 --2- 172.21.15.88:0/2808052016 >> v2:172.21.15.195:3300/0 conn(0x7f62bc0ccd00 0x7f62bc15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 -- 172.21.15.88:0/2808052016 >> 172.21.15.88:0/2808052016 conn(0x7f62bc0cd9f0 msgr2=0x7f62bc0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:04.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 -- 172.21.15.88:0/2808052016 shutdown_connections 2024-09-21T02:04:04.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.587+0000 7f62c204d640 1 -- 172.21.15.88:0/2808052016 wait complete. 2024-09-21T02:04:04.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.589+0000 7f62c204d640 1 Processor -- start 2024-09-21T02:04:04.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.589+0000 7f62c204d640 1 -- start start 2024-09-21T02:04:04.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62c204d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc0cad60 0x7f62bc1f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62c204d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc0cb170 0x7f62bc15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62c204d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0ccd00 0x7f62bc153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62c204d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc06d470 con 0x7f62bc0ccd00 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62c204d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc15c640 con 0x7f62bc0cad60 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62c204d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f62bc06d5f0 con 0x7f62bc0cb170 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0ccd00 0x7f62bc153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0ccd00 0x7f62bc153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52614/0 (socket says 172.21.15.88:52614) 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bbfff640 1 -- 172.21.15.88:0/3723821207 learned_addr learned my addr 172.21.15.88:0/3723821207 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bb7fe640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc0cad60 0x7f62bc1f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62baffd640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc0cb170 0x7f62bc15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bbfff640 1 -- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc0cb170 msgr2=0x7f62bc15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bbfff640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc0cb170 0x7f62bc15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bbfff640 1 -- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc0cad60 msgr2=0x7f62bc1f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.590+0000 7f62bbfff640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc0cad60 0x7f62bc1f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.591+0000 7f62bbfff640 1 -- 172.21.15.88:0/3723821207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc1f1c20 con 0x7f62bc0ccd00 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.591+0000 7f62bbfff640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0ccd00 0x7f62bc153340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62b402abe0 tx=0x7f62b4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.591+0000 7f62a3fff640 1 -- 172.21.15.88:0/3723821207 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b403e030 con 0x7f62bc0ccd00 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.591+0000 7f62a3fff640 1 -- 172.21.15.88:0/3723821207 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62b4003e40 con 0x7f62bc0ccd00 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.591+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc1f2620 con 0x7f62bc0ccd00 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.591+0000 7f62a3fff640 1 -- 172.21.15.88:0/3723821207 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4035040 con 0x7f62bc0ccd00 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.591+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc0c4c60 con 0x7f62bc0ccd00 2024-09-21T02:04:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.592+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62bc15f6d0 con 0x7f62bc0ccd00 2024-09-21T02:04:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.593+0000 7f62a3fff640 1 -- 172.21.15.88:0/3723821207 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62b4041030 con 0x7f62bc0ccd00 2024-09-21T02:04:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.593+0000 7f62a3fff640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a8078b90 0x7f62a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.593+0000 7f62a3fff640 1 -- 172.21.15.88:0/3723821207 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f62b40bb3f0 con 0x7f62bc0ccd00 2024-09-21T02:04:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.593+0000 7f62bb7fe640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a8078b90 0x7f62a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.594+0000 7f62bb7fe640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a8078b90 0x7f62a807b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62ac005fd0 tx=0x7f62ac0095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:04.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.597+0000 7f62a3fff640 1 -- 172.21.15.88:0/3723821207 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62bc15f6d0 con 0x7f62bc0ccd00 2024-09-21T02:04:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.774+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62bc06de60 con 0x7f62bc0ccd00 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.775+0000 7f62a3fff640 1 -- 172.21.15.88:0/3723821207 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f62bc1f1c20 con 0x7f62bc0ccd00 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:04.775 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":21,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:59.066086+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a8078b90 msgr2=0x7f62a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a8078b90 0x7f62a807b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62ac005fd0 tx=0x7f62ac0095e0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0ccd00 msgr2=0x7f62bc153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0ccd00 0x7f62bc153340 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62b402abe0 tx=0x7f62b4003910 comp rx=0 tx=0).stop 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 shutdown_connections 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a8078b90 0x7f62a807b0a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62bc0ccd00 0x7f62bc153340 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc0cb170 0x7f62bc15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 --2- 172.21.15.88:0/3723821207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc0cad60 0x7f62bc1f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 >> 172.21.15.88:0/3723821207 conn(0x7f62bc0cd9f0 msgr2=0x7f62bc1555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 shutdown_connections 2024-09-21T02:04:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:04.777+0000 7f62c204d640 1 -- 172.21.15.88:0/3723821207 wait complete. 2024-09-21T02:04:04.788 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T02:04:04.788 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T02:04:04.789 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T02:04:04.789 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T02:04:04.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T02:04:04.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T02:04:04.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T02:04:04.791 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T02:04:05.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.000+0000 7f47f240f640 1 -- 172.21.15.88:0/1757415730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 msgr2=0x7f47ec0c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f9a27cbd640 1 -- 172.21.15.88:0/1750059180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 msgr2=0x7f9a2006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f9a27cbd640 1 --2- 172.21.15.88:0/1750059180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 0x7f9a2006eab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a200cda30 tx=0x7f9a1002b6e0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.000+0000 7f47e3fff640 1 -- 172.21.15.88:0/1757415730 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4040070 con 0x7f47ec0c33a0 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.000+0000 7f47f240f640 1 --2- 172.21.15.88:0/1757415730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 0x7f47ec0c37b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f47e4005fd0 tx=0x7f47e4033120 comp rx=0 tx=0).stop 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f47f240f640 1 -- 172.21.15.88:0/1757415730 shutdown_connections 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f47f240f640 1 --2- 172.21.15.88:0/1757415730 >> v2:172.21.15.195:3300/0 conn(0x7f47ec0ccf10 0x7f47ec0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f47f240f640 1 --2- 172.21.15.88:0/1757415730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 0x7f47ec0c37b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f47f240f640 1 -- 172.21.15.88:0/1757415730 >> 172.21.15.88:0/1757415730 conn(0x7f47ec070b80 msgr2=0x7f47ec070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f9a27cbd640 1 -- 172.21.15.88:0/1750059180 shutdown_connections 2024-09-21T02:04:05.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f9a27cbd640 1 --2- 172.21.15.88:0/1750059180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 0x7f9a2006eab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f9a27cbd640 1 --2- 172.21.15.88:0/1750059180 >> v2:172.21.15.195:3300/0 conn(0x7f9a200c4890 0x7f9a20074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f9a27cbd640 1 -- 172.21.15.88:0/1750059180 >> 172.21.15.88:0/1750059180 conn(0x7f9a200ccb90 msgr2=0x7f9a200ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f47f240f640 1 -- 172.21.15.88:0/1757415730 shutdown_connections 2024-09-21T02:04:05.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.001+0000 7f47f240f640 1 -- 172.21.15.88:0/1757415730 wait complete. 2024-09-21T02:04:05.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.002+0000 7f9a27cbd640 1 -- 172.21.15.88:0/1750059180 shutdown_connections 2024-09-21T02:04:05.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.002+0000 7f9a27cbd640 1 -- 172.21.15.88:0/1750059180 wait complete. 2024-09-21T02:04:05.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.004+0000 7f47f240f640 1 Processor -- start 2024-09-21T02:04:05.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.004+0000 7f47f240f640 1 -- start start 2024-09-21T02:04:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.005+0000 7f47f240f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47ec074290 0x7f47ec201ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.005+0000 7f47f240f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 0x7f47ec0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.005+0000 7f47f240f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47ec0ccf10 0x7f47ec08c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.005+0000 7f47f240f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec0b2fa0 con 0x7f47ec0c33a0 2024-09-21T02:04:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.005+0000 7f47f240f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec0b2e20 con 0x7f47ec0ccf10 2024-09-21T02:04:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.005+0000 7f47f240f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec072bd0 con 0x7f47ec074290 2024-09-21T02:04:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f140d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47ec074290 0x7f47ec201ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f2d57fff640 1 -- 172.21.15.88:0/4237833434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d580749b0 msgr2=0x7f2d58073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f2d57fff640 1 --2- 172.21.15.88:0/4237833434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d580749b0 0x7f2d58073bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005fd0 tx=0x7f2d4802b880 comp rx=0 tx=0).stop 2024-09-21T02:04:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f1c0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47ec0ccf10 0x7f47ec08c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f140d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47ec074290 0x7f47ec201ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34676/0 (socket says 172.21.15.88:34676) 2024-09-21T02:04:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f1c0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47ec0ccf10 0x7f47ec08c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:49744/0 (socket says 172.21.15.88:49744) 2024-09-21T02:04:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f140d640 1 -- 172.21.15.88:0/2928389428 learned_addr learned my addr 172.21.15.88:0/2928389428 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f2d57fff640 1 -- 172.21.15.88:0/4237833434 shutdown_connections 2024-09-21T02:04:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f2d57fff640 1 --2- 172.21.15.88:0/4237833434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d58074130 0x7f2d58070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f2d57fff640 1 --2- 172.21.15.88:0/4237833434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d580749b0 0x7f2d58073bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f2d57fff640 1 -- 172.21.15.88:0/4237833434 >> 172.21.15.88:0/4237833434 conn(0x7f2d580cdd50 msgr2=0x7f2d5806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f140d640 1 -- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47ec0ccf10 msgr2=0x7f47ec08c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f0c0c640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 0x7f47ec0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f140d640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47ec0ccf10 0x7f47ec08c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f140d640 1 -- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 msgr2=0x7f47ec0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.006+0000 7f47f140d640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 0x7f47ec0cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f47f140d640 1 -- 172.21.15.88:0/2928389428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec205a40 con 0x7f47ec074290 2024-09-21T02:04:05.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f2d57fff640 1 -- 172.21.15.88:0/4237833434 shutdown_connections 2024-09-21T02:04:05.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f47f0c0c640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ec0c33a0 0x7f47ec0cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:05.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f2d57fff640 1 -- 172.21.15.88:0/4237833434 wait complete. 2024-09-21T02:04:05.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f47f140d640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47ec074290 0x7f47ec201ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f47e800b440 tx=0x7f47e800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.004+0000 7f9a27cbd640 1 Processor -- start 2024-09-21T02:04:05.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.005+0000 7f9a27cbd640 1 -- start start 2024-09-21T02:04:05.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a27cbd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 0x7f9a20153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8015020 con 0x7f47ec074290 2024-09-21T02:04:05.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e80119a0 con 0x7f47ec074290 2024-09-21T02:04:05.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8011df0 con 0x7f47ec074290 2024-09-21T02:04:05.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f47f240f640 1 -- 172.21.15.88:0/2928389428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec1687c0 con 0x7f47ec074290 2024-09-21T02:04:05.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7f47f240f640 1 -- 172.21.15.88:0/2928389428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec167020 con 0x7f47ec074290 2024-09-21T02:04:05.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7f3969339640 1 -- 172.21.15.88:0/1475852800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396415c580 msgr2=0x7f396415ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7f3969339640 1 --2- 172.21.15.88:0/1475852800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396415c580 0x7f396415ef80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3954009830 tx=0x7f395402a380 comp rx=0 tx=0).stop 2024-09-21T02:04:05.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a25a59640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 0x7f9a20153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2656285514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 msgr2=0x7fbd1c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2656285514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 0x7fbd1c0c7cb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd14005fd0 tx=0x7fbd1402acc0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a25a59640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 0x7f9a20153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52656/0 (socket says 172.21.15.88:52656) 2024-09-21T02:04:05.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a25a59640 1 -- 172.21.15.88:0/2738124113 learned_addr learned my addr 172.21.15.88:0/2738124113 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a27cbd640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a200c4480 0x7f9a200cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a27cbd640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a2014b4f0 0x7f9a201fb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f3969339640 1 -- 172.21.15.88:0/1475852800 shutdown_connections 2024-09-21T02:04:05.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f3969339640 1 --2- 172.21.15.88:0/1475852800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396415c580 0x7f396415ef80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f3969339640 1 --2- 172.21.15.88:0/1475852800 >> v2:172.21.15.195:3300/0 conn(0x7f3964155de0 0x7f396415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f3969339640 1 -- 172.21.15.88:0/1475852800 >> 172.21.15.88:0/1475852800 conn(0x7f39640720b0 msgr2=0x7f39640cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a25258640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a200c4480 0x7f9a200cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7f9a2625a640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a2014b4f0 0x7f9a201fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.007+0000 7f9a27cbd640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a200c7590 con 0x7f9a2006e6a0 2024-09-21T02:04:05.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7f9a27cbd640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20074630 con 0x7f9a2014b4f0 2024-09-21T02:04:05.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.008+0000 7f9a27cbd640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9a200c7710 con 0x7f9a200c4480 2024-09-21T02:04:05.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f47f240f640 1 -- 172.21.15.88:0/2928389428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47ec205620 con 0x7f47ec074290 2024-09-21T02:04:05.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f3969339640 1 -- 172.21.15.88:0/1475852800 shutdown_connections 2024-09-21T02:04:05.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f3969339640 1 -- 172.21.15.88:0/1475852800 wait complete. 2024-09-21T02:04:05.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f9a25a59640 1 -- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a200c4480 msgr2=0x7f9a200cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f9a25a59640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a200c4480 0x7f9a200cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2656285514 shutdown_connections 2024-09-21T02:04:05.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2656285514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 0x7fbd1c0c7cb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2656285514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd1c0c8ac0 0x7fbd1c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2656285514 >> 172.21.15.88:0/2656285514 conn(0x7fbd1c06d8f0 msgr2=0x7fbd1c0ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2656285514 shutdown_connections 2024-09-21T02:04:05.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f2d57fff640 1 Processor -- start 2024-09-21T02:04:05.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f9a25a59640 1 -- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a2014b4f0 msgr2=0x7f9a201fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f9a25a59640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a2014b4f0 0x7f9a201fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f9a25a59640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a20153390 con 0x7f9a2006e6a0 2024-09-21T02:04:05.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f9a25a59640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 0x7f9a20153f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800ba50 tx=0x7f9a1800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7f9a2625a640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a2014b4f0 0x7f9a201fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:05.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.009+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2656285514 wait complete. 2024-09-21T02:04:05.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.010+0000 7f2d57fff640 1 -- start start 2024-09-21T02:04:05.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.010+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18019070 con 0x7f9a2006e6a0 2024-09-21T02:04:05.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.010+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a1800ccf0 con 0x7f9a2006e6a0 2024-09-21T02:04:05.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.010+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a180146a0 con 0x7f9a2006e6a0 2024-09-21T02:04:05.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.010+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2973588792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca4077550 msgr2=0x7f4ca4075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.010+0000 7f4ca9ade640 1 --2- 172.21.15.88:0/2973588792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca4077550 0x7f4ca4075700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009860 tx=0x7f4c9802a580 comp rx=0 tx=0).stop 2024-09-21T02:04:05.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.010+0000 7f9a27cbd640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a20154460 con 0x7f9a2006e6a0 2024-09-21T02:04:05.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7f9a27cbd640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a20156cd0 con 0x7f9a2006e6a0 2024-09-21T02:04:05.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7ff99959a640 1 -- 172.21.15.88:0/101500764 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c031c30 con 0x7ff99c151d10 2024-09-21T02:04:05.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/101500764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c151d10 msgr2=0x7ff99c06e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7ff9a0dc3640 1 --2- 172.21.15.88:0/101500764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c151d10 0x7ff99c06e6a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff99c074b20 tx=0x7ff98c02a540 comp rx=0 tx=0).stop 2024-09-21T02:04:05.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/101500764 shutdown_connections 2024-09-21T02:04:05.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7ff9a0dc3640 1 --2- 172.21.15.88:0/101500764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c151d10 0x7ff99c06e6a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2973588792 shutdown_connections 2024-09-21T02:04:05.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7f4ca9ade640 1 --2- 172.21.15.88:0/2973588792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca4077550 0x7f4ca4075700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2973588792 >> 172.21.15.88:0/2973588792 conn(0x7f4ca406cf70 msgr2=0x7f4ca406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/101500764 >> 172.21.15.88:0/101500764 conn(0x7ff99c0ccb90 msgr2=0x7ff99c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47e8018020 con 0x7f47ec074290 2024-09-21T02:04:05.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.011+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/101500764 shutdown_connections 2024-09-21T02:04:05.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f3969339640 1 Processor -- start 2024-09-21T02:04:05.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d58074130 0x7f2d580c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580749b0 0x7f2d5813ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d58157bf0 0x7f2d5813f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d4e7fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580749b0 0x7f2d5813ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f47e1ffb640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f47d0078c90 0x7f47d007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2973588792 shutdown_connections 2024-09-21T02:04:05.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d4e7fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580749b0 0x7f2d5813ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34712/0 (socket says 172.21.15.88:34712) 2024-09-21T02:04:05.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7fbd20ad3640 1 Processor -- start 2024-09-21T02:04:05.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d4e7fc640 1 -- 172.21.15.88:0/561806431 learned_addr learned my addr 172.21.15.88:0/561806431 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7fbd20ad3640 1 -- start start 2024-09-21T02:04:05.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f3969339640 1 -- start start 2024-09-21T02:04:05.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d57fff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580c4630 con 0x7f2d58157bf0 2024-09-21T02:04:05.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d577fe640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d58157bf0 0x7f2d5813f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d57fff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d58074540 con 0x7f2d58074130 2024-09-21T02:04:05.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f2d57fff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d58068d00 con 0x7f2d580749b0 2024-09-21T02:04:05.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2973588792 wait complete. 2024-09-21T02:04:05.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f47f0c0c640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f47d0078c90 0x7f47d007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f51bdc97640 1 -- 172.21.15.88:0/2439647265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a0003a30 msgr2=0x7f51a0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f51bdc97640 1 --2- 172.21.15.88:0/2439647265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a0003a30 0x7f51a0003ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f51b0001dc0 tx=0x7f51a0005830 comp rx=0 tx=0).stop 2024-09-21T02:04:05.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f51bd496640 1 -- 172.21.15.88:0/2439647265 reap_dead start 2024-09-21T02:04:05.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f51bdc97640 1 -- 172.21.15.88:0/2439647265 shutdown_connections 2024-09-21T02:04:05.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f51bdc97640 1 -- 172.21.15.88:0/2439647265 >> 172.21.15.88:0/2439647265 conn(0x7f51b806cf70 msgr2=0x7f51b806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f47e809afa0 con 0x7f47ec074290 2024-09-21T02:04:05.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f9a27cbd640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a20157a20 con 0x7f9a2006e6a0 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d56ffd640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d58074130 0x7f2d580c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d4e7fc640 1 -- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d58074130 msgr2=0x7f2d580c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d4e7fc640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d58074130 0x7f2d580c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d4e7fc640 1 -- 172.21.15.88:0/561806431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d58157bf0 msgr2=0x7f2d5813f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d4e7fc640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d58157bf0 0x7f2d5813f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d4e7fc640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5813f640 con 0x7f2d580749b0 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d56ffd640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d58074130 0x7f2d580c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:05.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f47e1ffb640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7f47d0080f60 0x7f47d00833b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.012+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/101500764 wait complete. 2024-09-21T02:04:05.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f2d4e7fc640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580749b0 0x7f2d5813ebc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800a960 tx=0x7f2d480317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 --> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47e800de50 con 0x7f47d0080f60 2024-09-21T02:04:05.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.013+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f47e809b390 con 0x7f47ec074290 2024-09-21T02:04:05.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f47f1c0e640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7f47d0080f60 0x7f47d00833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48040070 con 0x7f2d580749b0 2024-09-21T02:04:05.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d48031bb0 con 0x7f2d580749b0 2024-09-21T02:04:05.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48036050 con 0x7f2d580749b0 2024-09-21T02:04:05.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f3969339640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3964155de0 0x7f39641512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f51bdc97640 1 -- 172.21.15.88:0/2439647265 shutdown_connections 2024-09-21T02:04:05.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f51bdc97640 1 -- 172.21.15.88:0/2439647265 wait complete. 2024-09-21T02:04:05.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7ff9a0dc3640 1 Processor -- start 2024-09-21T02:04:05.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7ff9a0dc3640 1 -- start start 2024-09-21T02:04:05.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9a1801c070 con 0x7f9a2006e6a0 2024-09-21T02:04:05.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f47f1c0e640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7f47d0080f60 0x7f47d00833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f3969339640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f396415c580 0x7f39641fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f3969339640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396415f4c0 0x7f39640ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f3969339640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39640c8dc0 con 0x7f396415f4c0 2024-09-21T02:04:05.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f3969339640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f396415bb20 con 0x7f396415c580 2024-09-21T02:04:05.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f3969339640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f39640b40d0 con 0x7f3964155de0 2024-09-21T02:04:05.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f2d57fff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d580c8c30 con 0x7f2d580749b0 2024-09-21T02:04:05.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.014+0000 7f2d57fff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d58150a50 con 0x7f2d580749b0 2024-09-21T02:04:05.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f4ca9ade640 1 Processor -- start 2024-09-21T02:04:05.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f47f0c0c640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f47d0078c90 0x7f47d007b1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47e403d000 tx=0x7f47e4004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== osd.3 v2:172.21.15.88:6816/723368200 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47e800de50 con 0x7f47d0080f60 2024-09-21T02:04:05.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3964155de0 0x7f39641512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3964155de0 0x7f39641512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34718/0 (socket says 172.21.15.88:34718) 2024-09-21T02:04:05.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 -- 172.21.15.88:0/1340616877 learned_addr learned my addr 172.21.15.88:0/1340616877 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7fbd20ad3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 0x7fbd1c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f39637fe640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396415f4c0 0x7f39640ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f2d57fff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d58157530 con 0x7f2d580749b0 2024-09-21T02:04:05.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f9a167fc640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9a04078b90 0x7f9a0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.015+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9a1809abb0 con 0x7f9a2006e6a0 2024-09-21T02:04:05.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f9a167fc640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f9a04080e60 0x7f9a040832b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f9a2625a640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f9a04080e60 0x7f9a040832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a1800ee50 con 0x7f9a04080e60 2024-09-21T02:04:05.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7fbd1a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 0x7fbd1c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7fbd1a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 0x7fbd1c0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52702/0 (socket says 172.21.15.88:52702) 2024-09-21T02:04:05.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7fbd1a59c640 1 -- 172.21.15.88:0/2869705816 learned_addr learned my addr 172.21.15.88:0/2869705816 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd1c0c8ac0 0x7fbd1c0746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 -- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f396415c580 msgr2=0x7f39641fa680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:04:05.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f396415c580 0x7f39641fa680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 -- 172.21.15.88:0/1340616877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396415f4c0 msgr2=0x7f39640ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f396415f4c0 0x7f39640ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f3962ffd640 1 -- 172.21.15.88:0/1340616877 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3964152060 con 0x7f3964155de0 2024-09-21T02:04:05.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9a1809f2a0 con 0x7f9a2006e6a0 2024-09-21T02:04:05.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f3962ffd640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3964155de0 0x7f39641512b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f395c00ea50 tx=0x7f395c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7ff9a0dc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c072bd0 0x7ff99c13e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f9a2625a640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f9a04080e60 0x7f9a040832b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd1c16c190 0x7fbd1c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c0c2140 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c00cdc0 con 0x7f3964155de0 2024-09-21T02:04:05.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c0c1fc0 con 0x7fbd1c0c8ac0 2024-09-21T02:04:05.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396415eef0 con 0x7f3964155de0 2024-09-21T02:04:05.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f9a25258640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9a04078b90 0x7f9a0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbd1c0c94c0 con 0x7fbd1c16c190 2024-09-21T02:04:05.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f395c011560 con 0x7f3964155de0 2024-09-21T02:04:05.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.016+0000 7f4ca9ade640 1 -- start start 2024-09-21T02:04:05.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f4ca9ade640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca4073d50 0x7f4ca40c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca9ade640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca4077550 0x7f4ca40cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d48043070 con 0x7f2d580749b0 2024-09-21T02:04:05.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f2d4ffff640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d38078c60 0x7f2d3807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7fbd1a59c640 1 -- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd1c16c190 msgr2=0x7fbd1c15a610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:04:05.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7fbd1a59c640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd1c16c190 0x7fbd1c15a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7fbd1a59c640 1 -- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd1c0c8ac0 msgr2=0x7fbd1c0746e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:04:05.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7fbd1a59c640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd1c0c8ac0 0x7fbd1c0746e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7fbd1a59c640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c0c6940 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c011990 con 0x7f3964155de0 2024-09-21T02:04:05.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.017+0000 7ff9a0dc3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff99c151900 0x7ff99c13a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7ff99a59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c072bd0 0x7ff99c13e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7ff99a59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c072bd0 0x7ff99c13e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:49790/0 (socket says 172.21.15.88:49790) 2024-09-21T02:04:05.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2d480bbc50 con 0x7f2d580749b0 2024-09-21T02:04:05.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f2d4ffff640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f2d38080eb0 0x7f2d38083300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d48038e50 con 0x7f2d38080eb0 2024-09-21T02:04:05.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2d48080100 con 0x7f2d580749b0 2024-09-21T02:04:05.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39641f9e80 con 0x7f3964155de0 2024-09-21T02:04:05.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f2d56ffd640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d38078c60 0x7f2d3807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7ff99a59c640 1 -- 172.21.15.88:0/3569961444 learned_addr learned my addr 172.21.15.88:0/3569961444 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f9a25258640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9a04078b90 0x7f9a0407b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005fd0 tx=0x7f9a1c005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== osd.2 v2:172.21.15.88:6808/1554825340 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a1800ee50 con 0x7f9a04080e60 2024-09-21T02:04:05.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f51bdc97640 1 Processor -- start 2024-09-21T02:04:05.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7ff9a0dc3640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99c151d10 0x7ff99c13ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f51bdc97640 1 -- start start 2024-09-21T02:04:05.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7ff999d9b640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff99c151900 0x7ff99c13a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c0c7590 con 0x7ff99c151900 2024-09-21T02:04:05.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f2d577fe640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f2d38080eb0 0x7f2d38083300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f2d56ffd640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d38078c60 0x7f2d3807b170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000a8d0 tx=0x7f2d40008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca9ade640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca40c2ac0 0x7f4ca40cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca9ade640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40c3780 con 0x7f4ca4077550 2024-09-21T02:04:05.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca37fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca4073d50 0x7f4ca40c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca2ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca4077550 0x7f4ca40cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca40c2ac0 0x7f4ca40cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca2ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca4077550 0x7f4ca40cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52716/0 (socket says 172.21.15.88:52716) 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca2ffd640 1 -- 172.21.15.88:0/2025165352 learned_addr learned my addr 172.21.15.88:0/2025165352 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca3fff640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca40c2ac0 0x7f4ca40cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34744/0 (socket says 172.21.15.88:34744) 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40c2640 con 0x7f4ca4073d50 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.018+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40c3a60 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f4ca3fff640 1 -- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca4073d50 msgr2=0x7f4ca40c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f4ca3fff640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ca4073d50 0x7f4ca40c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f4ca3fff640 1 -- 172.21.15.88:0/2025165352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca4077550 msgr2=0x7f4ca40cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f4ca3fff640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca4077550 0x7f4ca40cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f4ca3fff640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40c5ba0 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7ff99ad9d640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99c151d10 0x7ff99c13ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c074630 con 0x7ff99c072bd0 2024-09-21T02:04:05.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f395c011b30 con 0x7f3964155de0 2024-09-21T02:04:05.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39640c42f0 con 0x7f3964155de0 2024-09-21T02:04:05.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7fbd1a59c640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 0x7fbd1c0caca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b7b0 tx=0x7fbd0c00bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c019070 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f4ca3fff640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca40c2ac0 0x7f4ca40cc620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00c700 tx=0x7f4c9c00cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.019+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd0c00c810 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c0c7710 con 0x7ff99c151d10 2024-09-21T02:04:05.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c00cc60 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c017070 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f394b7fe640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3950078b90 0x7f395007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c0cb350 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c9c003c60 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c0ccf10 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f395c09b1e0 con 0x7f3964155de0 2024-09-21T02:04:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f39627fc640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3950078b90 0x7f395007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f394b7fe640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7f3950080de0 0x7f3950083230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 --> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3950083900 con 0x7f3950080de0 2024-09-21T02:04:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f395c0a2050 con 0x7f3964155de0 2024-09-21T02:04:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7ff99a59c640 1 -- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99c151d10 msgr2=0x7ff99c13ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0052d0 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7ff99a59c640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99c151d10 0x7ff99c13ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7ff99a59c640 1 -- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff99c151900 msgr2=0x7ff99c13a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7ff99a59c640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff99c151900 0x7ff99c13a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.020+0000 7ff99a59c640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c1574a0 con 0x7ff99c072bd0 2024-09-21T02:04:05.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7ff99a59c640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c072bd0 0x7ff99c13e130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c460 tx=0x7ff99000c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900052d0 con 0x7ff99c072bd0 2024-09-21T02:04:05.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff990005470 con 0x7ff99c072bd0 2024-09-21T02:04:05.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f51bdc97640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51b80b0be0 0x7f51b81f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990012890 con 0x7ff99c072bd0 2024-09-21T02:04:05.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f51bdc97640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51b80b2390 0x7f51b81f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f51bcc95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51b80b0be0 0x7f51b81f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f51bcc95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51b80b0be0 0x7f51b81f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52724/0 (socket says 172.21.15.88:52724) 2024-09-21T02:04:05.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f51bcc95640 1 -- 172.21.15.88:0/1943115491 learned_addr learned my addr 172.21.15.88:0/1943115491 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f39637fe640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7f3950080de0 0x7f3950083230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca413efc0 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca4153c60 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7f51bdc97640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51b815a840 0x7f51b81483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8072bf0 con 0x7f51b80b0be0 2024-09-21T02:04:05.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd1c0c3310 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f39637fe640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7f3950080de0 0x7f3950083230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f39627fc640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3950078b90 0x7f395007b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3954034000 tx=0x7f3954004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f51b815b880 con 0x7f51b815a840 2024-09-21T02:04:05.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f51b7fff640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51b80b2390 0x7f51b81f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f51b80c8740 con 0x7f51b80b2390 2024-09-21T02:04:05.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f51bd496640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51b815a840 0x7f51b81483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c156170 con 0x7ff99c072bd0 2024-09-21T02:04:05.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.021+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c151140 con 0x7ff99c072bd0 2024-09-21T02:04:05.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f2d577fe640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f2d38080eb0 0x7f2d38083300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.022+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ca4075700 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff99c157940 con 0x7ff99c072bd0 2024-09-21T02:04:05.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51bcc95640 1 -- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51b80b2390 msgr2=0x7f51b81f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== osd.4 v2:172.21.15.195:6818/2209751664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3950083900 con 0x7f3950080de0 2024-09-21T02:04:05.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51bcc95640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51b80b2390 0x7f51b81f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51bcc95640 1 -- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51b815a840 msgr2=0x7f51b81483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51bcc95640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51b815a840 0x7f51b81483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51bcc95640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b80af860 con 0x7f51b80b0be0 2024-09-21T02:04:05.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51b7fff640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51b80b2390 0x7f51b81f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:04:05.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== osd.5 v2:172.21.15.195:6826/3321376084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d48038e50 con 0x7f2d38080eb0 2024-09-21T02:04:05.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51bcc95640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51b80b0be0 0x7f51b81f2c70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f51b002ac40 tx=0x7f51b0003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0006240 con 0x7f51b80b0be0 2024-09-21T02:04:05.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51b0006400 con 0x7f51b80b0be0 2024-09-21T02:04:05.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0006850 con 0x7f51b80b0be0 2024-09-21T02:04:05.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.023+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b80afb40 con 0x7f51b80b0be0 2024-09-21T02:04:05.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.024+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b80b0100 con 0x7f51b80b0be0 2024-09-21T02:04:05.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.024+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9900035f0 con 0x7ff99c072bd0 2024-09-21T02:04:05.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.025+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd0c01c070 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.025+0000 7ff98affd640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff97c078c90 0x7ff97c07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.025+0000 7ff999d9b640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff97c078c90 0x7ff97c07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.025+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff99009a7a0 con 0x7ff99c072bd0 2024-09-21T02:04:05.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.025+0000 7ff999d9b640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff97c078c90 0x7ff97c07b1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff99c150830 tx=0x7ff994009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7fbd0affd640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbcfc07d070 0x7fbcfc07f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7ff98affd640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7ff97c080f60 0x7ff97c0833b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff99001ae50 con 0x7ff97c080f60 2024-09-21T02:04:05.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff9900a1050 con 0x7ff99c072bd0 2024-09-21T02:04:05.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7fbd19d9b640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbcfc07d070 0x7fbcfc07f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7f519effd640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51b8073e10 con 0x7f51b80b0be0 2024-09-21T02:04:05.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7ff99ad9d640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7ff97c080f60 0x7ff97c0833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd0c09a8a0 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7fbd19d9b640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbcfc07d070 0x7fbcfc07f580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd14035000 tx=0x7fbd14004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.027+0000 7fbd0affd640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbcfc085340 0x7fbcfc087790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.027+0000 7fbd1ad9d640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbcfc085340 0x7fbcfc087790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.027+0000 7ff99ad9d640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7ff97c080f60 0x7ff97c0833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.026+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c9c01a070 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.027+0000 7f4c8bfff640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4c90078c10 0x7f4c9007b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.027+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4c9c09a8f0 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.027+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbcfc087e60 con 0x7fbcfc085340 2024-09-21T02:04:05.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f4ca37fe640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4c90078c10 0x7f4c9007b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f4c8bfff640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f4c90080e80 0x7f4c900832d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c900839a0 con 0x7f4c90080e80 2024-09-21T02:04:05.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4c9c09add0 con 0x7f4ca40c2ac0 2024-09-21T02:04:05.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.027+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51b0033480 con 0x7f51b80b0be0 2024-09-21T02:04:05.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f51b57fa640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f51a4078bc0 0x7f51a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f51b00bada0 con 0x7f51b80b0be0 2024-09-21T02:04:05.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f51b57fa640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7f51a4080db0 0x7f51a4083200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== osd.1 v2:172.21.15.88:6824/2879208543 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff99001ae50 con 0x7ff97c080f60 2024-09-21T02:04:05.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f4ca2ffd640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f4c90080e80 0x7f4c900832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51b003cec0 con 0x7f51a4080db0 2024-09-21T02:04:05.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f51b00bd440 con 0x7f51b80b0be0 2024-09-21T02:04:05.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7f51b7fff640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f51a4078bc0 0x7f51a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7fbd1ad9d640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbcfc085340 0x7fbcfc087790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.028+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbd0c09ac90 con 0x7fbd1c0c78a0 2024-09-21T02:04:05.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.029+0000 7f4ca37fe640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4c90078c10 0x7f4c9007b120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c98034000 tx=0x7f4c98004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.029+0000 7f4ca2ffd640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f4c90080e80 0x7f4c900832d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.029+0000 7f51b7fff640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f51a4078bc0 0x7f51a407b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0114f0 tx=0x7f51ac0093d0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.029+0000 7f51bd496640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7f51a4080db0 0x7f51a4083200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.029+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== osd.0 v2:172.21.15.88:6800/1826444869 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcfc087e60 con 0x7fbcfc085340 2024-09-21T02:04:05.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.030+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== osd.6 v2:172.21.15.195:6802/1905698167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c900839a0 con 0x7f4c90080e80 2024-09-21T02:04:05.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.030+0000 7f51bd496640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7f51a4080db0 0x7f51a4083200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.032+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== osd.7 v2:172.21.15.195:6810/985762803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51b003cec0 con 0x7f51a4080db0 2024-09-21T02:04:05.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.043+0000 7f47f240f640 1 -- 172.21.15.88:0/2928389428 --> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47ec168570 con 0x7f47d0080f60 2024-09-21T02:04:05.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.044+0000 7f47e1ffb640 1 -- 172.21.15.88:0/2928389428 <== osd.3 v2:172.21.15.88:6816/723368200 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f47ec168570 con 0x7f47d0080f60 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.044+0000 7f47cf7fe640 1 -- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7f47d0080f60 msgr2=0x7f47d00833b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.044+0000 7f47cf7fe640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7f47d0080f60 0x7f47d00833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.045+0000 7f47cf7fe640 1 -- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f47d0078c90 msgr2=0x7f47d007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.045+0000 7f47cf7fe640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f47d0078c90 0x7f47d007b1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47e403d000 tx=0x7f47e4004ab0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.045+0000 7f47cf7fe640 1 -- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47ec074290 msgr2=0x7f47ec201ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.045+0000 7f47cf7fe640 1 --2- 172.21.15.88:0/2928389428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47ec074290 0x7f47ec201ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f47e800b440 tx=0x7f47e800b900 comp rx=0 tx=0).stop 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.045+0000 7f47f1c0e640 1 -- 172.21.15.88:0/2928389428 reap_dead start 2024-09-21T02:04:05.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.046+0000 7f47cf7fe640 1 -- 172.21.15.88:0/2928389428 shutdown_connections 2024-09-21T02:04:05.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.046+0000 7f47cf7fe640 1 -- 172.21.15.88:0/2928389428 >> 172.21.15.88:0/2928389428 conn(0x7f47ec070b80 msgr2=0x7f47ec168cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.046+0000 7f47cf7fe640 1 -- 172.21.15.88:0/2928389428 shutdown_connections 2024-09-21T02:04:05.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.046+0000 7f47cf7fe640 1 -- 172.21.15.88:0/2928389428 wait complete. 2024-09-21T02:04:05.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.048+0000 7f9a27cbd640 1 -- 172.21.15.88:0/2738124113 --> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a200c9070 con 0x7f9a04080e60 2024-09-21T02:04:05.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.048+0000 7f9a167fc640 1 -- 172.21.15.88:0/2738124113 <== osd.2 v2:172.21.15.88:6808/1554825340 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9a200c9070 con 0x7f9a04080e60 2024-09-21T02:04:05.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.049+0000 7f9a03fff640 1 -- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f9a04080e60 msgr2=0x7f9a040832b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.049+0000 7f9a03fff640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f9a04080e60 0x7f9a040832b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a03fff640 1 -- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9a04078b90 msgr2=0x7f9a0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a03fff640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9a04078b90 0x7f9a0407b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005fd0 tx=0x7f9a1c005850 comp rx=0 tx=0).stop 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a03fff640 1 -- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 msgr2=0x7f9a20153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a03fff640 1 --2- 172.21.15.88:0/2738124113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a2006e6a0 0x7f9a20153f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800ba50 tx=0x7f9a1800bf10 comp rx=0 tx=0).stop 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a2625a640 1 -- 172.21.15.88:0/2738124113 reap_dead start 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a03fff640 1 -- 172.21.15.88:0/2738124113 shutdown_connections 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a03fff640 1 -- 172.21.15.88:0/2738124113 >> 172.21.15.88:0/2738124113 conn(0x7f9a200ccb90 msgr2=0x7f9a200c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.050+0000 7f9a03fff640 1 -- 172.21.15.88:0/2738124113 shutdown_connections 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.051+0000 7f9a03fff640 1 -- 172.21.15.88:0/2738124113 wait complete. 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.052+0000 7f2d57fff640 1 -- 172.21.15.88:0/561806431 --> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d58069260 con 0x7f2d38080eb0 2024-09-21T02:04:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.052+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 --> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f39640c42f0 con 0x7f3950080de0 2024-09-21T02:04:05.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.053+0000 7f2d4ffff640 1 -- 172.21.15.88:0/561806431 <== osd.5 v2:172.21.15.195:6826/3321376084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d58069260 con 0x7f2d38080eb0 2024-09-21T02:04:05.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.053+0000 7f394b7fe640 1 -- 172.21.15.88:0/1340616877 <== osd.4 v2:172.21.15.195:6818/2209751664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f39640c42f0 con 0x7f3950080de0 2024-09-21T02:04:05.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.053+0000 7f2d4d7fa640 1 -- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f2d38080eb0 msgr2=0x7f2d38083300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.053+0000 7f2d4d7fa640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f2d38080eb0 0x7f2d38083300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.053+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7f3950080de0 msgr2=0x7f3950083230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f3969339640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7f3950080de0 0x7f3950083230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 -- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d38078c60 msgr2=0x7f2d3807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d38078c60 0x7f2d3807b170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000a8d0 tx=0x7f2d40008040 comp rx=0 tx=0).stop 2024-09-21T02:04:05.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 -- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580749b0 msgr2=0x7f2d5813ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 --2- 172.21.15.88:0/561806431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d580749b0 0x7f2d5813ebc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800a960 tx=0x7f2d480317b0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d577fe640 1 -- 172.21.15.88:0/561806431 reap_dead start 2024-09-21T02:04:05.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 -- 172.21.15.88:0/561806431 shutdown_connections 2024-09-21T02:04:05.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 -- 172.21.15.88:0/561806431 >> 172.21.15.88:0/561806431 conn(0x7f2d580cdd50 msgr2=0x7f2d581547f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3950078b90 msgr2=0x7f395007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 -- 172.21.15.88:0/561806431 shutdown_connections 2024-09-21T02:04:05.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f2d4d7fa640 1 -- 172.21.15.88:0/561806431 wait complete. 2024-09-21T02:04:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f3969339640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3950078b90 0x7f395007b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3954034000 tx=0x7f3954004730 comp rx=0 tx=0).stop 2024-09-21T02:04:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.054+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3964155de0 msgr2=0x7f39641512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.055+0000 7f3969339640 1 --2- 172.21.15.88:0/1340616877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3964155de0 0x7f39641512b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f395c00ea50 tx=0x7f395c00ef10 comp rx=0 tx=0).stop 2024-09-21T02:04:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.055+0000 7f39637fe640 1 -- 172.21.15.88:0/1340616877 reap_dead start 2024-09-21T02:04:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.055+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 shutdown_connections 2024-09-21T02:04:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.055+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 >> 172.21.15.88:0/1340616877 conn(0x7f39640720b0 msgr2=0x7f3964159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.055+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 shutdown_connections 2024-09-21T02:04:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.056+0000 7f3969339640 1 -- 172.21.15.88:0/1340616877 wait complete. 2024-09-21T02:04:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.057+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 --> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff99c0c9070 con 0x7ff97c080f60 2024-09-21T02:04:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.057+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 --> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd1c0c8450 con 0x7fbcfc085340 2024-09-21T02:04:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.058+0000 7ff98affd640 1 -- 172.21.15.88:0/3569961444 <== osd.1 v2:172.21.15.88:6824/2879208543 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff99c0c9070 con 0x7ff97c080f60 2024-09-21T02:04:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.058+0000 7fbd0affd640 1 -- 172.21.15.88:0/2869705816 <== osd.0 v2:172.21.15.88:6800/1826444869 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbd1c0c8450 con 0x7fbcfc085340 2024-09-21T02:04:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.058+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbcfc085340 msgr2=0x7fbcfc087790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.058+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbcfc085340 0x7fbcfc087790 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.059+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7ff97c080f60 msgr2=0x7ff97c0833b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.059+0000 7ff9a0dc3640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7ff97c080f60 0x7ff97c0833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.059+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff97c078c90 msgr2=0x7ff97c07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.059+0000 7ff9a0dc3640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ff97c078c90 0x7ff97c07b1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff99c150830 tx=0x7ff994009040 comp rx=0 tx=0).stop 2024-09-21T02:04:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.059+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c072bd0 msgr2=0x7ff99c13e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.059+0000 7ff9a0dc3640 1 --2- 172.21.15.88:0/3569961444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff99c072bd0 0x7ff99c13e130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c460 tx=0x7ff99000c920 comp rx=0 tx=0).stop 2024-09-21T02:04:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.059+0000 7ff99ad9d640 1 -- 172.21.15.88:0/3569961444 reap_dead start 2024-09-21T02:04:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbcfc07d070 msgr2=0x7fbcfc07f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbcfc07d070 0x7fbcfc07f580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd14035000 tx=0x7fbd14004ab0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 --> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ca41450a0 con 0x7f4c90080e80 2024-09-21T02:04:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 shutdown_connections 2024-09-21T02:04:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 >> 172.21.15.88:0/3569961444 conn(0x7ff99c0ccb90 msgr2=0x7ff99c13ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7f519effd640 1 -- 172.21.15.88:0/1943115491 --> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51b80c8300 con 0x7f51a4080db0 2024-09-21T02:04:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 msgr2=0x7fbd1c0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7fbd20ad3640 1 --2- 172.21.15.88:0/2869705816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1c0c78a0 0x7fbd1c0caca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b7b0 tx=0x7fbd0c00bc70 comp rx=0 tx=0).stop 2024-09-21T02:04:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7fbd1ad9d640 1 -- 172.21.15.88:0/2869705816 reap_dead start 2024-09-21T02:04:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 shutdown_connections 2024-09-21T02:04:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7ff9a0dc3640 1 -- 172.21.15.88:0/3569961444 wait complete. 2024-09-21T02:04:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7f51b57fa640 1 -- 172.21.15.88:0/1943115491 <== osd.7 v2:172.21.15.195:6810/985762803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f51b80c8300 con 0x7f51a4080db0 2024-09-21T02:04:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.060+0000 7f4c8bfff640 1 -- 172.21.15.88:0/2025165352 <== osd.6 v2:172.21.15.195:6802/1905698167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ca41450a0 con 0x7f4c90080e80 2024-09-21T02:04:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7f51a4080db0 msgr2=0x7f51a4083200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7f51a4080db0 0x7f51a4083200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 shutdown_connections 2024-09-21T02:04:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 >> 172.21.15.88:0/2869705816 conn(0x7fbd1c06d8f0 msgr2=0x7fbd1c1638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f51a4078bc0 msgr2=0x7f51a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f51a4078bc0 0x7f51a407b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0114f0 tx=0x7f51ac0093d0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51b80b0be0 msgr2=0x7f51b81f2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 --2- 172.21.15.88:0/1943115491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51b80b0be0 0x7f51b81f2c70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f51b002ac40 tx=0x7f51b0003ab0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bd496640 1 -- 172.21.15.88:0/1943115491 reap_dead start 2024-09-21T02:04:05.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f4c90080e80 msgr2=0x7f4c900832d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f4ca9ade640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f4c90080e80 0x7f4c900832d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 shutdown_connections 2024-09-21T02:04:05.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.061+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 >> 172.21.15.88:0/1943115491 conn(0x7f51b806cf70 msgr2=0x7f51b80c3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 shutdown_connections 2024-09-21T02:04:05.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7fbd20ad3640 1 -- 172.21.15.88:0/2869705816 wait complete. 2024-09-21T02:04:05.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4c90078c10 msgr2=0x7f4c9007b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f4ca9ade640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4c90078c10 0x7f4c9007b120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c98034000 tx=0x7f4c98004730 comp rx=0 tx=0).stop 2024-09-21T02:04:05.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca40c2ac0 msgr2=0x7f4ca40cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f4ca9ade640 1 --2- 172.21.15.88:0/2025165352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ca40c2ac0 0x7f4ca40cc620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00c700 tx=0x7f4c9c00cbc0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 shutdown_connections 2024-09-21T02:04:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f51bdc97640 1 -- 172.21.15.88:0/1943115491 wait complete. 2024-09-21T02:04:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f4ca3fff640 1 -- 172.21.15.88:0/2025165352 reap_dead start 2024-09-21T02:04:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.062+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 shutdown_connections 2024-09-21T02:04:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.063+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 >> 172.21.15.88:0/2025165352 conn(0x7f4ca406cf70 msgr2=0x7f4ca415c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.063+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 shutdown_connections 2024-09-21T02:04:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.064+0000 7f4ca9ade640 1 -- 172.21.15.88:0/2025165352 wait complete. 2024-09-21T02:04:05.092 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2024-09-21T02:04:05.093 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T02:04:05.094 INFO:teuthology.orchestra.run.smithi088.stdout:64424509445 2024-09-21T02:04:05.094 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T02:04:05.095 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T02:04:05.095 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T02:04:05.096 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T02:04:05.096 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T02:04:05.097 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2024-09-21T02:04:05.097 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T02:04:05.098 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-09-21T02:04:05.098 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T02:04:05.099 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-09-21T02:04:05.099 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T02:04:05.099 INFO:teuthology.orchestra.run.smithi088.stdout:64424509445 2024-09-21T02:04:05.100 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T02:04:05.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 -- 172.21.15.88:0/1564104955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80b1e60 msgr2=0x7fa9e8155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 --2- 172.21.15.88:0/1564104955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80b1e60 0x7fa9e8155c70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80cdba0 tx=0x7fa9e002a750 comp rx=0 tx=0).stop 2024-09-21T02:04:05.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 -- 172.21.15.88:0/1564104955 shutdown_connections 2024-09-21T02:04:05.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 --2- 172.21.15.88:0/1564104955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e80c28d0 0x7fa9e80c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 --2- 172.21.15.88:0/1564104955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80b1e60 0x7fa9e8155c70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 -- 172.21.15.88:0/1564104955 >> 172.21.15.88:0/1564104955 conn(0x7fa9e806d8f0 msgr2=0x7fa9e80ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 -- 172.21.15.88:0/1564104955 shutdown_connections 2024-09-21T02:04:05.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.349+0000 7fa9eea94640 1 -- 172.21.15.88:0/1564104955 wait complete. 2024-09-21T02:04:05.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.351+0000 7f12ab7e4640 1 -- 172.21.15.88:0/2610589940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 msgr2=0x7f12a40736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.351+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/2610589940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a40736d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009830 tx=0x7f12a002a560 comp rx=0 tx=0).stop 2024-09-21T02:04:05.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.351+0000 7f12ab7e4640 1 -- 172.21.15.88:0/2610589940 shutdown_connections 2024-09-21T02:04:05.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.351+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/2610589940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a40736d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.351+0000 7f12ab7e4640 1 -- 172.21.15.88:0/2610589940 >> 172.21.15.88:0/2610589940 conn(0x7f12a40ccb90 msgr2=0x7f12a40ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.351+0000 7f12ab7e4640 1 -- 172.21.15.88:0/2610589940 shutdown_connections 2024-09-21T02:04:05.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.351+0000 7f12ab7e4640 1 -- 172.21.15.88:0/2610589940 wait complete. 2024-09-21T02:04:05.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.352+0000 7fa9eea94640 1 Processor -- start 2024-09-21T02:04:05.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c0159a640 1 -- 172.21.15.88:0/377186740 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc02fe30 con 0x7f3c0406d2a0 2024-09-21T02:04:05.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f0744507640 1 -- 172.21.15.88:0/2787754179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f073c1642e0 msgr2=0x7f073c166700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f0744507640 1 --2- 172.21.15.88:0/2787754179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f073c1642e0 0x7f073c166700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0734005fd0 tx=0x7f073402a890 comp rx=0 tx=0).stop 2024-09-21T02:04:05.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 -- 172.21.15.88:0/377186740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 msgr2=0x7f3c040cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/377186740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 0x7f3c040cc230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c0408f0b0 tx=0x7f3bfc02aa30 comp rx=0 tx=0).stop 2024-09-21T02:04:05.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 -- 172.21.15.88:0/377186740 shutdown_connections 2024-09-21T02:04:05.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/377186740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 0x7f3c040cc230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/377186740 >> v2:172.21.15.88:3300/0 conn(0x7f3c040b6eb0 0x7f3c0406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 -- 172.21.15.88:0/377186740 >> 172.21.15.88:0/377186740 conn(0x7f3c040cda90 msgr2=0x7f3c040cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 -- 172.21.15.88:0/377186740 shutdown_connections 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f0744507640 1 -- 172.21.15.88:0/2787754179 shutdown_connections 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f0744507640 1 --2- 172.21.15.88:0/2787754179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f073c1642e0 0x7f073c166700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7f0744507640 1 --2- 172.21.15.88:0/2787754179 >> v2:172.21.15.88:3300/0 conn(0x7f073c0c6230 0x7f073c0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7f0744507640 1 -- 172.21.15.88:0/2787754179 >> 172.21.15.88:0/2787754179 conn(0x7f073c06f230 msgr2=0x7f073c06f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa640389640 1 -- 172.21.15.88:0/2208386204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638152840 msgr2=0x7fa638162ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa640389640 1 --2- 172.21.15.88:0/2208386204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638152840 0x7fa638162ff0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa630005fd0 tx=0x7fa63002a800 comp rx=0 tx=0).stop 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa640389640 1 -- 172.21.15.88:0/2208386204 shutdown_connections 2024-09-21T02:04:05.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa640389640 1 --2- 172.21.15.88:0/2208386204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638152840 0x7fa638162ff0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa640389640 1 --2- 172.21.15.88:0/2208386204 >> v2:172.21.15.88:3300/0 conn(0x7fa638072e40 0x7fa638158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa640389640 1 -- 172.21.15.88:0/2208386204 >> 172.21.15.88:0/2208386204 conn(0x7fa63806d8f0 msgr2=0x7fa6380ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa640389640 1 -- 172.21.15.88:0/2208386204 shutdown_connections 2024-09-21T02:04:05.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7f0744507640 1 -- 172.21.15.88:0/2787754179 shutdown_connections 2024-09-21T02:04:05.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7f0744507640 1 -- 172.21.15.88:0/2787754179 wait complete. 2024-09-21T02:04:05.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7f12ab7e4640 1 Processor -- start 2024-09-21T02:04:05.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7f12ab7e4640 1 -- start start 2024-09-21T02:04:05.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.353+0000 7f3c08ad5640 1 -- 172.21.15.88:0/377186740 wait complete. 2024-09-21T02:04:05.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.352+0000 7fa9eea94640 1 -- start start 2024-09-21T02:04:05.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.354+0000 7fa9eea94640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e80b18f0 0x7fa9e8152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9ec830640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e80b18f0 0x7fa9e8152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f0a01339640 1 -- 172.21.15.88:0/4184757001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc0c6230 msgr2=0x7f09fc0ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f0a01339640 1 --2- 172.21.15.88:0/4184757001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc0c6230 0x7f09fc0ce450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009900 tx=0x7f09ec02a6a0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f0a01339640 1 -- 172.21.15.88:0/4184757001 shutdown_connections 2024-09-21T02:04:05.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f0a01339640 1 --2- 172.21.15.88:0/4184757001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc155a80 0x7f09fc074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f0a01339640 1 --2- 172.21.15.88:0/4184757001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc0c6230 0x7f09fc0ce450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f0a01339640 1 -- 172.21.15.88:0/4184757001 >> 172.21.15.88:0/4184757001 conn(0x7f09fc06f230 msgr2=0x7f09fc06f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9eea94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e80b1e60 0x7fa9e81fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9eea94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80c28d0 0x7fa9e81fa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9eea94640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80c1ec0 con 0x7fa9e80b18f0 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9ec830640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e80b18f0 0x7fa9e8152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52780/0 (socket says 172.21.15.88:52780) 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9ec830640 1 -- 172.21.15.88:0/4109359707 learned_addr learned my addr 172.21.15.88:0/4109359707 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9eea94640 1 -- 172.21.15.88:0/4109359707 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80c9590 con 0x7fa9e80c28d0 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9eea94640 1 -- 172.21.15.88:0/4109359707 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e80c8390 con 0x7fa9e80b1e60 2024-09-21T02:04:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa640389640 1 -- 172.21.15.88:0/2208386204 wait complete. 2024-09-21T02:04:05.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f0a01339640 1 -- 172.21.15.88:0/4184757001 shutdown_connections 2024-09-21T02:04:05.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9e7fff640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e80b1e60 0x7fa9e81fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f112bfff640 1 -- 172.21.15.88:0/1118447101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1118004d90 msgr2=0x7f1118005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f112bfff640 1 --2- 172.21.15.88:0/1118447101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1118004d90 0x7f1118005220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f112c072ba0 tx=0x7f1118005830 comp rx=0 tx=0).stop 2024-09-21T02:04:05.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7f112b7fe640 1 -- 172.21.15.88:0/1118447101 reap_dead start 2024-09-21T02:04:05.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f0a01339640 1 -- 172.21.15.88:0/4184757001 wait complete. 2024-09-21T02:04:05.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.355+0000 7fa9ed031640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80c28d0 0x7fa9e81fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7fa9ec830640 1 -- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e80b1e60 msgr2=0x7fa9e81fa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7fa9ec830640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e80b1e60 0x7fa9e81fa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7fa9ec830640 1 -- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80c28d0 msgr2=0x7fa9e81fa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7fa9ec830640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80c28d0 0x7fa9e81fa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7fa9ec830640 1 -- 172.21.15.88:0/4109359707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e81fb070 con 0x7fa9e80b18f0 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7fa9ec830640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e80b18f0 0x7fa9e8152230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009b30 tx=0x7fa9e0032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7fa9e57fa640 1 -- 172.21.15.88:0/4109359707 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0041020 con 0x7fa9e80b18f0 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7fa9e57fa640 1 -- 172.21.15.88:0/4109359707 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9e0037040 con 0x7fa9e80b18f0 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7fa9e57fa640 1 -- 172.21.15.88:0/4109359707 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0036070 con 0x7fa9e80b18f0 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f0744507640 1 Processor -- start 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f112bfff640 1 -- 172.21.15.88:0/1118447101 shutdown_connections 2024-09-21T02:04:05.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f112bfff640 1 -- 172.21.15.88:0/1118447101 >> 172.21.15.88:0/1118447101 conn(0x7f112c06cf70 msgr2=0x7f112c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f3c08ad5640 1 Processor -- start 2024-09-21T02:04:05.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f3c08ad5640 1 -- start start 2024-09-21T02:04:05.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f14bdf61640 1 -- 172.21.15.88:0/1828348785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14a80037a0 msgr2=0x7f14a8003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f14bdf61640 1 --2- 172.21.15.88:0/1828348785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14a80037a0 0x7f14a8003bb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14b000ef40 tx=0x7f14a8005830 comp rx=0 tx=0).stop 2024-09-21T02:04:05.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f12ab7e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12a4072bd0 0x7f12a41f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f12ab7e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a4148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.356+0000 7f12ab7e4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a40c5240 0x7f12a41e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a9580640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12a4072bd0 0x7f12a41f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a9580640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12a4072bd0 0x7f12a41f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34844/0 (socket says 172.21.15.88:34844) 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a8d7f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a4148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a8d7f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a4148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:49886/0 (socket says 172.21.15.88:49886) 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a9580640 1 -- 172.21.15.88:0/856271801 learned_addr learned my addr 172.21.15.88:0/856271801 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40c5650 con 0x7f12a40c5240 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40c63e0 con 0x7f12a40c4090 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12a4158bf0 con 0x7f12a4072bd0 2024-09-21T02:04:05.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f0744507640 1 -- start start 2024-09-21T02:04:05.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f14bd760640 1 -- 172.21.15.88:0/1828348785 reap_dead start 2024-09-21T02:04:05.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f112bfff640 1 -- 172.21.15.88:0/1118447101 shutdown_connections 2024-09-21T02:04:05.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a8d7f640 1 -- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12a4072bd0 msgr2=0x7f12a41f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a8d7f640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12a4072bd0 0x7f12a41f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f112bfff640 1 -- 172.21.15.88:0/1118447101 wait complete. 2024-09-21T02:04:05.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7fa640389640 1 Processor -- start 2024-09-21T02:04:05.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a8d7f640 1 -- 172.21.15.88:0/856271801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a40c5240 msgr2=0x7f12a41e7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7fa9eea94640 1 -- 172.21.15.88:0/4109359707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e8075580 con 0x7fa9e80b18f0 2024-09-21T02:04:05.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7fa640389640 1 -- start start 2024-09-21T02:04:05.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f3c08ad5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 0x7f3c040c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a8d7f640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a40c5240 0x7f12a41e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.357+0000 7f12a8d7f640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a41f6320 con 0x7f12a40c4090 2024-09-21T02:04:05.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f3c08ad5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c040b6eb0 0x7f3c041f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f12a8d7f640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a4148800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f129400c4d0 tx=0x7f129400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f3c08ad5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c040c3d70 0x7f3c0415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f1299ffb640 1 -- 172.21.15.88:0/856271801 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294017070 con 0x7f12a40c4090 2024-09-21T02:04:05.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f1299ffb640 1 -- 172.21.15.88:0/856271801 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12940035f0 con 0x7f12a40c4090 2024-09-21T02:04:05.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f1299ffb640 1 -- 172.21.15.88:0/856271801 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294003a40 con 0x7f12a40c4090 2024-09-21T02:04:05.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c01d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c040b6eb0 0x7f3c041f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c01d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c040b6eb0 0x7f3c041f93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34860/0 (socket says 172.21.15.88:34860) 2024-09-21T02:04:05.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a40cb200 con 0x7f12a40c4090 2024-09-21T02:04:05.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.358+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a40cb4e0 con 0x7f12a40c4090 2024-09-21T02:04:05.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0a01339640 1 Processor -- start 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0744507640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f073c0c6230 0x7f073c14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0744507640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c155a80 0x7f073c14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0744507640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f073c1642e0 0x7f073c142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0744507640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0cdf30 con 0x7f073c155a80 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0744507640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0cdc50 con 0x7f073c1642e0 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c02d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c040c3d70 0x7f3c0415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c08ad5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c040cbd10 con 0x7f3c040c3d70 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c02d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c040c3d70 0x7f3c0415a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52788/0 (socket says 172.21.15.88:52788) 2024-09-21T02:04:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c02d9d640 1 -- 172.21.15.88:0/3638231555 learned_addr learned my addr 172.21.15.88:0/3638231555 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3c040cba30 con 0x7f3c0406d2a0 2024-09-21T02:04:05.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0406dcd0 con 0x7f3c040b6eb0 2024-09-21T02:04:05.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0a01339640 1 -- start start 2024-09-21T02:04:05.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f0744507640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f073c0c5e60 con 0x7f073c0c6230 2024-09-21T02:04:05.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0742aa4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f073c1642e0 0x7f073c142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c155a80 0x7f073c14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.359+0000 7f3c0259c640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 0x7f3c040c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c155a80 0x7f073c14bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52792/0 (socket says 172.21.15.88:52792) 2024-09-21T02:04:05.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 -- 172.21.15.88:0/670416036 learned_addr learned my addr 172.21.15.88:0/670416036 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f112bfff640 1 Processor -- start 2024-09-21T02:04:05.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f3c02d9d640 1 -- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c040b6eb0 msgr2=0x7f3c041f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f3c02d9d640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c040b6eb0 0x7f3c041f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f3c02d9d640 1 -- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 msgr2=0x7f3c040c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f3c02d9d640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 0x7f3c040c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f3c02d9d640 1 -- 172.21.15.88:0/3638231555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c041f9930 con 0x7f3c040c3d70 2024-09-21T02:04:05.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7fa640389640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa638072e40 0x7fa638150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f112bfff640 1 -- start start 2024-09-21T02:04:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7fa640389640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638073f40 0x7fa638154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 -- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f073c0c6230 msgr2=0x7f073c14b000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f073c0c6230 0x7f073c14b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 -- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f073c1642e0 msgr2=0x7f073c142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f073c1642e0 0x7f073c142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7fa640389640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa638152840 0x7fa6381f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f14bdf61640 1 -- 172.21.15.88:0/1828348785 shutdown_connections 2024-09-21T02:04:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f14bdf61640 1 -- 172.21.15.88:0/1828348785 >> 172.21.15.88:0/1828348785 conn(0x7f14b0050f50 msgr2=0x7f14b0051360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f0741aa2640 1 -- 172.21.15.88:0/670416036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c14a560 con 0x7f073c155a80 2024-09-21T02:04:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa640389640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa638075940 con 0x7fa638152840 2024-09-21T02:04:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638073f40 0x7fa638154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63e125640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa638072e40 0x7fa638150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638073f40 0x7fa638154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34864/0 (socket says 172.21.15.88:34864) 2024-09-21T02:04:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 -- 172.21.15.88:0/4064816831 learned_addr learned my addr 172.21.15.88:0/4064816831 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f3c02d9d640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c040c3d70 0x7f3c0415a4e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009bf0 tx=0x7f3bfc003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f14bdf61640 1 -- 172.21.15.88:0/1828348785 shutdown_connections 2024-09-21T02:04:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63e926640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa638152840 0x7fa6381f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f14bdf61640 1 -- 172.21.15.88:0/1828348785 wait complete. 2024-09-21T02:04:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380c9710 con 0x7fa638072e40 2024-09-21T02:04:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380c1ec0 con 0x7fa638073f40 2024-09-21T02:04:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa9eea94640 1 -- 172.21.15.88:0/4109359707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e8074a60 con 0x7fa9e80b18f0 2024-09-21T02:04:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f0741aa2640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c155a80 0x7f073c14bda0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f072c00b3f0 tx=0x7f072c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.360+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a40736d0 con 0x7f12a40c4090 2024-09-21T02:04:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f1299ffb640 1 -- 172.21.15.88:0/856271801 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f129401a070 con 0x7f12a40c4090 2024-09-21T02:04:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f3bf2ffd640 1 -- 172.21.15.88:0/3638231555 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc033050 con 0x7f3c040c3d70 2024-09-21T02:04:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f3bf2ffd640 1 -- 172.21.15.88:0/3638231555 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bfc0044d0 con 0x7f3c040c3d70 2024-09-21T02:04:05.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f3bf2ffd640 1 -- 172.21.15.88:0/3638231555 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc038450 con 0x7f3c040c3d70 2024-09-21T02:04:05.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f1299ffb640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1288078c60 0x7f128807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f1299ffb640 1 -- 172.21.15.88:0/856271801 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f129409a150 con 0x7f12a40c4090 2024-09-21T02:04:05.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c041f6ac0 con 0x7f3c040c3d70 2024-09-21T02:04:05.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 -- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa638072e40 msgr2=0x7fa638150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.362+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c040d0790 con 0x7f3c040c3d70 2024-09-21T02:04:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa638072e40 0x7fa638150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 -- 172.21.15.88:0/4064816831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa638152840 msgr2=0x7fa6381f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.362+0000 7f12a9580640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1288078c60 0x7f128807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa638152840 0x7fa6381f9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.361+0000 7fa63d924640 1 -- 172.21.15.88:0/4064816831 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa638154790 con 0x7fa638073f40 2024-09-21T02:04:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.362+0000 7fa9e57fa640 1 -- 172.21.15.88:0/4109359707 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9e0044020 con 0x7fa9e80b18f0 2024-09-21T02:04:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.362+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e8076dd0 con 0x7fa9e80b18f0 2024-09-21T02:04:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7fa63d924640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638073f40 0x7fa638154250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa62800b3e0 tx=0x7fa62800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.362+0000 7f12a9580640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1288078c60 0x7f128807b170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12a00095f0 tx=0x7f12a0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f112bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112c143990 0x7f112c1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f112bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112c1f92b0 0x7f112c0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f112bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c1575b0 0x7f112c1413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f0732ffd640 1 -- 172.21.15.88:0/670416036 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c015020 con 0x7f073c155a80 2024-09-21T02:04:05.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f0732ffd640 1 -- 172.21.15.88:0/670416036 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f072c011920 con 0x7f073c155a80 2024-09-21T02:04:05.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f0732ffd640 1 -- 172.21.15.88:0/670416036 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c011d30 con 0x7f073c155a80 2024-09-21T02:04:05.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f0744507640 1 -- 172.21.15.88:0/670416036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c1523c0 con 0x7f073c155a80 2024-09-21T02:04:05.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f112bfff640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f112c0c92c0 con 0x7f112c1575b0 2024-09-21T02:04:05.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f112bfff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f112c0cd590 con 0x7f112c143990 2024-09-21T02:04:05.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c040c96e0 con 0x7f3c040c3d70 2024-09-21T02:04:05.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f112bfff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f112c157f30 con 0x7f112c1f92b0 2024-09-21T02:04:05.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f0744507640 1 -- 172.21.15.88:0/670416036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c14a040 con 0x7f073c155a80 2024-09-21T02:04:05.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112c1f92b0 0x7f112c0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112c1f92b0 0x7f112c0cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:34880/0 (socket says 172.21.15.88:34880) 2024-09-21T02:04:05.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 -- 172.21.15.88:0/2810536342 learned_addr learned my addr 172.21.15.88:0/2810536342 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 -- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112c143990 msgr2=0x7f112c1583c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:05.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f0a01339640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc074d40 0x7f09fc153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112c143990 0x7f112c1583c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 -- 172.21.15.88:0/2810536342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c1575b0 msgr2=0x7f112c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c1575b0 0x7f112c1413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 -- 172.21.15.88:0/2810536342 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c158900 con 0x7f112c1f92b0 2024-09-21T02:04:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f0a01339640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc0c6230 0x7f09fc0d1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7fa9e57fa640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa9d4078c90 0x7fa9d407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.363+0000 7f11227fc640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112c1f92b0 0x7f112c0cbcc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f111400c7d0 tx=0x7f111400cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f0a01339640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc155a80 0x7f09fc157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f1128ff9640 1 -- 172.21.15.88:0/2810536342 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114017070 con 0x7f112c1f92b0 2024-09-21T02:04:05.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f09fa7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc0c6230 0x7f09fc0d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7fa9e57fa640 1 -- 172.21.15.88:0/4109359707 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa9e00c0060 con 0x7fa9e80b18f0 2024-09-21T02:04:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f14bdf61640 1 Processor -- start 2024-09-21T02:04:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f09fa7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc0c6230 0x7f09fc0d1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:49912/0 (socket says 172.21.15.88:49912) 2024-09-21T02:04:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f1128ff9640 1 -- 172.21.15.88:0/2810536342 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11140035f0 con 0x7f112c1f92b0 2024-09-21T02:04:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f1128ff9640 1 -- 172.21.15.88:0/2810536342 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114003a40 con 0x7f112c1f92b0 2024-09-21T02:04:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f09faffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc074d40 0x7f09fc153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f09faffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc074d40 0x7f09fc153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52822/0 (socket says 172.21.15.88:52822) 2024-09-21T02:04:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f09faffd640 1 -- 172.21.15.88:0/2007110396 learned_addr learned my addr 172.21.15.88:0/2007110396 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f112bfff640 1 -- 172.21.15.88:0/2810536342 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c150ed0 con 0x7f112c1f92b0 2024-09-21T02:04:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f09fb7fe640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc155a80 0x7f09fc157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f14bdf61640 1 -- start start 2024-09-21T02:04:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc0cdf30 con 0x7f09fc074d40 2024-09-21T02:04:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc0cdc50 con 0x7f09fc0c6230 2024-09-21T02:04:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc0c5e60 con 0x7f09fc155a80 2024-09-21T02:04:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7f112bfff640 1 -- 172.21.15.88:0/2810536342 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c143520 con 0x7f112c1f92b0 2024-09-21T02:04:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7f09faffd640 1 -- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc155a80 msgr2=0x7f09fc157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7fa62effd640 1 -- 172.21.15.88:0/4064816831 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628015020 con 0x7fa638073f40 2024-09-21T02:04:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7fa62effd640 1 -- 172.21.15.88:0/4064816831 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa628011920 con 0x7fa638073f40 2024-09-21T02:04:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7fa62effd640 1 -- 172.21.15.88:0/4064816831 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628011d70 con 0x7fa638073f40 2024-09-21T02:04:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6381fa480 con 0x7fa638073f40 2024-09-21T02:04:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6380c6d30 con 0x7fa638073f40 2024-09-21T02:04:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7f09faffd640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc155a80 0x7f09fc157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7f09faffd640 1 -- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc0c6230 msgr2=0x7f09fc0d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7f09faffd640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc0c6230 0x7f09fc0d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7f09faffd640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc154490 con 0x7f09fc074d40 2024-09-21T02:04:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.364+0000 7fa9e7fff640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa9d4078c90 0x7fa9d407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7fa9e7fff640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa9d4078c90 0x7fa9d407b1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9e806e7e0 tx=0x7fa9dc009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0718005290 con 0x7f073c155a80 2024-09-21T02:04:05.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.365+0000 7f09faffd640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc074d40 0x7f09fc153310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09ec034040 tx=0x7f09ec003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bdf61640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14a80037a0 0x7f14b00d7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1104005290 con 0x7f112c1f92b0 2024-09-21T02:04:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bdf61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b00cf660 0x7f14b00cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bdf61640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b00cfa70 0x7f14b00ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bdf61640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0008500 con 0x7f14b00cfa70 2024-09-21T02:04:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bdf61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14b004ce60 con 0x7f14b00cf660 2024-09-21T02:04:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bdf61640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f14b00dfaf0 con 0x7f14a80037a0 2024-09-21T02:04:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bcf5f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14a80037a0 0x7f14b00d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bd760640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b00cfa70 0x7f14b00ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bd760640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b00cfa70 0x7f14b00ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52826/0 (socket says 172.21.15.88:52826) 2024-09-21T02:04:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14bd760640 1 -- 172.21.15.88:0/3180370633 learned_addr learned my addr 172.21.15.88:0/3180370633 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7f14b7fff640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b00cf660 0x7f14b00cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.366+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6380c67c0 con 0x7fa638073f40 2024-09-21T02:04:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f14bd760640 1 -- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14a80037a0 msgr2=0x7f14b00d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f14bd760640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14a80037a0 0x7f14b00d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f14bd760640 1 -- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b00cf660 msgr2=0x7f14b00cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f14bd760640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b00cf660 0x7f14b00cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f1128ff9640 1 -- 172.21.15.88:0/2810536342 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f111401a070 con 0x7f112c1f92b0 2024-09-21T02:04:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f14bd760640 1 -- 172.21.15.88:0/3180370633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b00d6710 con 0x7f14b00cfa70 2024-09-21T02:04:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f14bd760640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b00cfa70 0x7f14b00ccde0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14b8000b60 tx=0x7f14b8051970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f1128ff9640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f110c078c10 0x7f110c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f1128ff9640 1 -- 172.21.15.88:0/2810536342 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f111409a2a0 con 0x7f112c1f92b0 2024-09-21T02:04:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.367+0000 7f112affd640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f110c078c10 0x7f110c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.368+0000 7fa62effd640 1 -- 172.21.15.88:0/4064816831 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa628018020 con 0x7fa638073f40 2024-09-21T02:04:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.368+0000 7f1299ffb640 1 -- 172.21.15.88:0/856271801 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1294062600 con 0x7f12a40c4090 2024-09-21T02:04:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.368+0000 7f112affd640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f110c078c10 0x7f110c07b120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f111c003670 tx=0x7f111c006c30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.368+0000 7fa62effd640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa61c078c10 0x7fa61c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.369+0000 7fa62effd640 1 -- 172.21.15.88:0/4064816831 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa62809a9e0 con 0x7fa638073f40 2024-09-21T02:04:05.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.369+0000 7fa63e125640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa61c078c10 0x7fa61c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.369+0000 7fa63e125640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa61c078c10 0x7fa61c07b120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa634005fd0 tx=0x7fa63400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.370+0000 7fa9e57fa640 1 -- 172.21.15.88:0/4109359707 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9e8076dd0 con 0x7fa9e80b18f0 2024-09-21T02:04:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.371+0000 7f14b57fa640 1 -- 172.21.15.88:0/3180370633 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00a610 con 0x7f14b00cfa70 2024-09-21T02:04:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.371+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b00d6c90 con 0x7f14b00cfa70 2024-09-21T02:04:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.371+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b00dde60 con 0x7f14b00cfa70 2024-09-21T02:04:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.371+0000 7f09e37fe640 1 -- 172.21.15.88:0/2007110396 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec03e070 con 0x7f09fc074d40 2024-09-21T02:04:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.371+0000 7f09e37fe640 1 -- 172.21.15.88:0/2007110396 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09ec004a90 con 0x7f09fc074d40 2024-09-21T02:04:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.371+0000 7f14b57fa640 1 -- 172.21.15.88:0/3180370633 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b8091070 con 0x7f14b00cfa70 2024-09-21T02:04:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.372+0000 7f14b57fa640 1 -- 172.21.15.88:0/3180370633 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8075a70 con 0x7f14b00cfa70 2024-09-21T02:04:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.372+0000 7f3bf2ffd640 1 -- 172.21.15.88:0/3638231555 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3bfc036070 con 0x7f3c040c3d70 2024-09-21T02:04:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f09e37fe640 1 -- 172.21.15.88:0/2007110396 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec035040 con 0x7f09fc074d40 2024-09-21T02:04:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc0d0920 con 0x7f09fc074d40 2024-09-21T02:04:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f0732ffd640 1 -- 172.21.15.88:0/670416036 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f072c018020 con 0x7f073c155a80 2024-09-21T02:04:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f1128ff9640 1 -- 172.21.15.88:0/2810536342 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1114062800 con 0x7f112c1f92b0 2024-09-21T02:04:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1498005290 con 0x7f14b00cfa70 2024-09-21T02:04:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f3bf2ffd640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3be4078c60 0x7f3be407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc0d0f20 con 0x7f09fc074d40 2024-09-21T02:04:05.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f3bf2ffd640 1 -- 172.21.15.88:0/3638231555 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3bfc0bc350 con 0x7f3c040c3d70 2024-09-21T02:04:05.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f0732ffd640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0720078c60 0x7f072007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f3c0259c640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3be4078c60 0x7f3be407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.373+0000 7f0732ffd640 1 -- 172.21.15.88:0/670416036 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f072c09b140 con 0x7f073c155a80 2024-09-21T02:04:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.374+0000 7f07422a3640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0720078c60 0x7f072007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.374+0000 7f3c0259c640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3be4078c60 0x7f3be407b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4000f60 tx=0x7f3bf4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.374+0000 7f07422a3640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0720078c60 0x7f072007b170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f07380058c0 tx=0x7f0738005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.375+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc155330 con 0x7f09fc074d40 2024-09-21T02:04:05.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.377+0000 7f3bf2ffd640 1 -- 172.21.15.88:0/3638231555 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c040c96e0 con 0x7f3c040c3d70 2024-09-21T02:04:05.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.377+0000 7fa62effd640 1 -- 172.21.15.88:0/4064816831 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa62801f020 con 0x7fa638073f40 2024-09-21T02:04:05.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.387+0000 7f0732ffd640 1 -- 172.21.15.88:0/670416036 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f072c01f020 con 0x7f073c155a80 2024-09-21T02:04:05.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.390+0000 7f14b57fa640 1 -- 172.21.15.88:0/3180370633 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f14b8074050 con 0x7f14b00cfa70 2024-09-21T02:04:05.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.391+0000 7f09e37fe640 1 -- 172.21.15.88:0/2007110396 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09ec041070 con 0x7f09fc074d40 2024-09-21T02:04:05.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.391+0000 7f14b57fa640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f149c078b90 0x7f149c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.391+0000 7f09e37fe640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:05.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.391+0000 7f09e37fe640 1 -- 172.21.15.88:0/2007110396 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f09ec0bbbf0 con 0x7f09fc074d40 2024-09-21T02:04:05.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.391+0000 7f14b57fa640 1 -- 172.21.15.88:0/3180370633 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f14b8116100 con 0x7f14b00cfa70 2024-09-21T02:04:05.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.392+0000 7f14bcf5f640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f149c078b90 0x7f149c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.392+0000 7f09fa7fc640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:05.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.392+0000 7f14bcf5f640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f149c078b90 0x7f149c07b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00a5e0 tx=0x7f14ac005eb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.393+0000 7f09fa7fc640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0ce710 tx=0x7f09f000b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:05.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.400+0000 7f09e37fe640 1 -- 172.21.15.88:0/2007110396 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09ec083a50 con 0x7f09fc074d40 2024-09-21T02:04:05.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.409+0000 7f14b57fa640 1 -- 172.21.15.88:0/3180370633 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f14b809d030 con 0x7f14b00cfa70 2024-09-21T02:04:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.788+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa638158dc0 con 0x7fa638073f40 2024-09-21T02:04:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.788+0000 7fa62effd640 1 -- 172.21.15.88:0/4064816831 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa628062940 con 0x7fa638073f40 2024-09-21T02:04:05.789 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-09-21T02:04:05.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.792+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa61c078c10 msgr2=0x7fa61c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.792+0000 7fa640389640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa61c078c10 0x7fa61c07b120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa634005fd0 tx=0x7fa63400b2e0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.792+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638073f40 msgr2=0x7fa638154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.792+0000 7fa640389640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638073f40 0x7fa638154250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa62800b3e0 tx=0x7fa62800b8a0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 shutdown_connections 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa61c078c10 0x7fa61c07b120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa638152840 0x7fa6381f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa638073f40 0x7fa638154250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 --2- 172.21.15.88:0/4064816831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa638072e40 0x7fa638150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 >> 172.21.15.88:0/4064816831 conn(0x7fa63806d8f0 msgr2=0x7fa6380ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 shutdown_connections 2024-09-21T02:04:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.793+0000 7fa640389640 1 -- 172.21.15.88:0/4064816831 wait complete. 2024-09-21T02:04:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.794+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1104005850 con 0x7f112c1f92b0 2024-09-21T02:04:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.794+0000 7f1128ff9640 1 -- 172.21.15.88:0/2810536342 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1114062100 con 0x7f112c1f92b0 2024-09-21T02:04:05.795 INFO:teuthology.orchestra.run.smithi088.stdout:64424509445 2024-09-21T02:04:05.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f110c078c10 msgr2=0x7f110c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f12a41589d0 con 0x7f12a40c4090 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f110c078c10 0x7f110c07b120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f111c003670 tx=0x7f111c006c30 comp rx=0 tx=0).stop 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112c1f92b0 msgr2=0x7f112c0cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112c1f92b0 0x7f112c0cbcc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f111400c7d0 tx=0x7f111400cc90 comp rx=0 tx=0).stop 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 shutdown_connections 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f110c078c10 0x7f110c07b120 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c1575b0 0x7f112c1413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112c1f92b0 0x7f112c0cbcc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 --2- 172.21.15.88:0/2810536342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112c143990 0x7f112c1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3c0406cd60 con 0x7f3c040c3d70 2024-09-21T02:04:05.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.798+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 >> 172.21.15.88:0/2810536342 conn(0x7f112c06cf70 msgr2=0x7f112c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.799+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa9e815a6f0 con 0x7fa9e80b18f0 2024-09-21T02:04:05.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.799+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 shutdown_connections 2024-09-21T02:04:05.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.799+0000 7f1121ffb640 1 -- 172.21.15.88:0/2810536342 wait complete. 2024-09-21T02:04:05.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.799+0000 7f1299ffb640 1 -- 172.21.15.88:0/856271801 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1294061f00 con 0x7f12a40c4090 2024-09-21T02:04:05.801 INFO:teuthology.orchestra.run.smithi088.stdout:64424509444 2024-09-21T02:04:05.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.799+0000 7fa9e57fa640 1 -- 172.21.15.88:0/4109359707 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa9e815a6f0 con 0x7fa9e80b18f0 2024-09-21T02:04:05.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.799+0000 7f3bf2ffd640 1 -- 172.21.15.88:0/3638231555 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3bfc084240 con 0x7f3c040c3d70 2024-09-21T02:04:05.802 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2024-09-21T02:04:05.802 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T02:04:05.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1288078c60 msgr2=0x7f128807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1288078c60 0x7f128807b170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12a00095f0 tx=0x7f12a0004730 comp rx=0 tx=0).stop 2024-09-21T02:04:05.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 msgr2=0x7f12a4148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a4148800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f129400c4d0 tx=0x7f129400c990 comp rx=0 tx=0).stop 2024-09-21T02:04:05.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 shutdown_connections 2024-09-21T02:04:05.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1288078c60 0x7f128807b170 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a40c5240 0x7f12a41e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa9d4078c90 msgr2=0x7fa9d407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa9d4078c90 0x7fa9d407b1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9e806e7e0 tx=0x7fa9dc009060 comp rx=0 tx=0).stop 2024-09-21T02:04:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e80b18f0 msgr2=0x7fa9e8152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e80b18f0 0x7fa9e8152230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009b30 tx=0x7fa9e0032b20 comp rx=0 tx=0).stop 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12a40c4090 0x7f12a4148800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 --2- 172.21.15.88:0/856271801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12a4072bd0 0x7f12a41f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 shutdown_connections 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa9d4078c90 0x7fa9d407b1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9e80c28d0 0x7fa9e81fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9e80b1e60 0x7fa9e81fa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 --2- 172.21.15.88:0/4109359707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e80b18f0 0x7fa9e8152230 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009b30 tx=0x7fa9e0032b20 comp rx=0 tx=0).stop 2024-09-21T02:04:05.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 >> 172.21.15.88:0/4109359707 conn(0x7fa9e806d8f0 msgr2=0x7fa9e81597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 >> 172.21.15.88:0/856271801 conn(0x7f12a40ccb90 msgr2=0x7f12a40c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 shutdown_connections 2024-09-21T02:04:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.803+0000 7f12ab7e4640 1 -- 172.21.15.88:0/856271801 wait complete. 2024-09-21T02:04:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 shutdown_connections 2024-09-21T02:04:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7fa9ceffd640 1 -- 172.21.15.88:0/4109359707 wait complete. 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3be4078c60 msgr2=0x7f3be407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3be4078c60 0x7f3be407b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4000f60 tx=0x7f3bf4009040 comp rx=0 tx=0).stop 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c040c3d70 msgr2=0x7f3c0415a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c040c3d70 0x7f3c0415a4e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009bf0 tx=0x7f3bfc003ef0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 shutdown_connections 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3be4078c60 0x7f3be407b170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c040c3d70 0x7f3c0415a4e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c040b6eb0 0x7f3c041f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 --2- 172.21.15.88:0/3638231555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c0406d2a0 0x7f3c040c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 >> 172.21.15.88:0/3638231555 conn(0x7f3c040cda90 msgr2=0x7f3c04158630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 shutdown_connections 2024-09-21T02:04:05.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.804+0000 7f3c08ad5640 1 -- 172.21.15.88:0/3638231555 wait complete. 2024-09-21T02:04:05.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.812+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0718005850 con 0x7f073c155a80 2024-09-21T02:04:05.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.812+0000 7f0732ffd640 1 -- 172.21.15.88:0/670416036 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f072c063680 con 0x7f073c155a80 2024-09-21T02:04:05.812 INFO:teuthology.orchestra.run.smithi088.stdout:64424509445 2024-09-21T02:04:05.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-09-21T02:04:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0720078c60 msgr2=0x7f072007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0720078c60 0x7f072007b170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f07380058c0 tx=0x7f0738005850 comp rx=0 tx=0).stop 2024-09-21T02:04:05.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c155a80 msgr2=0x7f073c14bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c155a80 0x7f073c14bda0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f072c00b3f0 tx=0x7f072c00b8b0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 shutdown_connections 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0720078c60 0x7f072007b170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f073c1642e0 0x7f073c142910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c155a80 0x7f073c14bda0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 --2- 172.21.15.88:0/670416036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f073c0c6230 0x7f073c14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 >> 172.21.15.88:0/670416036 conn(0x7f073c06f230 msgr2=0x7f073c1597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.816+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 shutdown_connections 2024-09-21T02:04:05.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.817+0000 7f0730ff9640 1 -- 172.21.15.88:0/670416036 wait complete. 2024-09-21T02:04:05.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-09-21T02:04:05.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.822+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1498005850 con 0x7f14b00cfa70 2024-09-21T02:04:05.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.822+0000 7f14b57fa640 1 -- 172.21.15.88:0/3180370633 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14b809d030 con 0x7f14b00cfa70 2024-09-21T02:04:05.822 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-09-21T02:04:05.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.7 2024-09-21T02:04:05.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2024-09-21T02:04:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.825+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f149c078b90 msgr2=0x7f149c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.825+0000 7f14bdf61640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f149c078b90 0x7f149c07b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00a5e0 tx=0x7f14ac005eb0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.825+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b00cfa70 msgr2=0x7f14b00ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.827 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-09-21T02:04:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.825+0000 7f14bdf61640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b00cfa70 0x7f14b00ccde0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14b8000b60 tx=0x7f14b8051970 comp rx=0 tx=0).stop 2024-09-21T02:04:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 shutdown_connections 2024-09-21T02:04:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f149c078b90 0x7f149c07b0a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b00cfa70 0x7f14b00ccde0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b00cf660 0x7f14b00cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 --2- 172.21.15.88:0/3180370633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14a80037a0 0x7f14b00d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 >> 172.21.15.88:0/3180370633 conn(0x7f14b0050f50 msgr2=0x7f14b0051360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 shutdown_connections 2024-09-21T02:04:05.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.826+0000 7f14bdf61640 1 -- 172.21.15.88:0/3180370633 wait complete. 2024-09-21T02:04:05.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.828+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f09fc0cbf50 con 0x7f09fc074d40 2024-09-21T02:04:05.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.828+0000 7f09e37fe640 1 -- 172.21.15.88:0/2007110396 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f09ec088480 con 0x7f09fc074d40 2024-09-21T02:04:05.829 INFO:teuthology.orchestra.run.smithi088.stdout:64424509444 2024-09-21T02:04:05.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 msgr2=0x7f09e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0ce710 tx=0x7f09f000b2e0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc074d40 msgr2=0x7f09fc153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:05.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc074d40 0x7f09fc153310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09ec034040 tx=0x7f09ec003b90 comp rx=0 tx=0).stop 2024-09-21T02:04:05.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 shutdown_connections 2024-09-21T02:04:05.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09fc155a80 0x7f09fc157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09fc0c6230 0x7f09fc0d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 --2- 172.21.15.88:0/2007110396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc074d40 0x7f09fc153310 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:05.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 >> 172.21.15.88:0/2007110396 conn(0x7f09fc06f230 msgr2=0x7f09fc1597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:05.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 shutdown_connections 2024-09-21T02:04:05.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:05.831+0000 7f0a01339640 1 -- 172.21.15.88:0/2007110396 wait complete. 2024-09-21T02:04:05.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-09-21T02:04:05.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-09-21T02:04:05.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.2 2024-09-21T02:04:06.814 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T02:04:06.822 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T02:04:06.827 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T02:04:06.827 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T02:04:06.828 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T02:04:06.834 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T02:04:06.839 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T02:04:06.843 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T02:04:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.004+0000 7fd6a5a39640 1 -- 172.21.15.88:0/338664674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00cd440 msgr2=0x7fd6a0074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.004+0000 7fd6a5a39640 1 --2- 172.21.15.88:0/338664674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00cd440 0x7fd6a0074370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd6900108b0 tx=0x7fd69002e3a0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.004+0000 7fd6a5a39640 1 -- 172.21.15.88:0/338664674 shutdown_connections 2024-09-21T02:04:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.004+0000 7fd6a5a39640 1 --2- 172.21.15.88:0/338664674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6a00748b0 0x7fd6a0074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.004+0000 7fd6a5a39640 1 --2- 172.21.15.88:0/338664674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00cd440 0x7fd6a0074370 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.005+0000 7fd6a5a39640 1 -- 172.21.15.88:0/338664674 >> 172.21.15.88:0/338664674 conn(0x7fd6a006cf70 msgr2=0x7fd6a006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.005+0000 7fd6a5a39640 1 -- 172.21.15.88:0/338664674 shutdown_connections 2024-09-21T02:04:07.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.005+0000 7fd6a5a39640 1 -- 172.21.15.88:0/338664674 wait complete. 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.008+0000 7fd6a5a39640 1 Processor -- start 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.008+0000 7fd6a5a39640 1 -- start start 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a5a39640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00748b0 0x7fd6a0167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a4a37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00748b0 0x7fd6a0167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a4a37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00748b0 0x7fd6a0167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52850/0 (socket says 172.21.15.88:52850) 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a4a37640 1 -- 172.21.15.88:0/769315780 learned_addr learned my addr 172.21.15.88:0/769315780 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a5a39640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6a00cd030 0x7fd6a0165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a5a39640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a00cd440 0x7fd6a00b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a5a39640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a00c8300 con 0x7fd6a00748b0 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a5a39640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0073f20 con 0x7fd6a00cd440 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.011+0000 7fd6a5a39640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a0072bf0 con 0x7fd6a00cd030 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.012+0000 7fd6a4a37640 1 -- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6a00cd030 msgr2=0x7fd6a0165300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.012+0000 7fd6a4a37640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6a00cd030 0x7fd6a0165300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.012+0000 7fd6a4a37640 1 -- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a00cd440 msgr2=0x7fd6a00b1a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.012+0000 7fd6a4a37640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a00cd440 0x7fd6a00b1a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.012+0000 7fd6a4a37640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a0165840 con 0x7fd6a00748b0 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.012+0000 7fd6a4a37640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00748b0 0x7fd6a0167fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c800 tx=0x7fd68800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.013+0000 7fd69d7fa640 1 -- 172.21.15.88:0/769315780 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688017070 con 0x7fd6a00748b0 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.013+0000 7fd69d7fa640 1 -- 172.21.15.88:0/769315780 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd688005270 con 0x7fd6a00748b0 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.013+0000 7fd69d7fa640 1 -- 172.21.15.88:0/769315780 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68800a640 con 0x7fd6a00748b0 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.013+0000 7fd6a5a39640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a02215a0 con 0x7fd6a00748b0 2024-09-21T02:04:07.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.013+0000 7fd6a5a39640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a0164340 con 0x7fd6a00748b0 2024-09-21T02:04:07.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.018+0000 7fd6a5a39640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd684005290 con 0x7fd6a00748b0 2024-09-21T02:04:07.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.020+0000 7fd69d7fa640 1 -- 172.21.15.88:0/769315780 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd68801a070 con 0x7fd6a00748b0 2024-09-21T02:04:07.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.020+0000 7fd69d7fa640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd680078c90 0x7fd68007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.021+0000 7fd69d7fa640 1 -- 172.21.15.88:0/769315780 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd68809a2d0 con 0x7fd6a00748b0 2024-09-21T02:04:07.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.021+0000 7fd69ffff640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd680078c90 0x7fd68007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.029+0000 7fd69d7fa640 1 -- 172.21.15.88:0/769315780 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd688062780 con 0x7fd6a00748b0 2024-09-21T02:04:07.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.042+0000 7fd69ffff640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd680078c90 0x7fd68007b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd690038000 tx=0x7fd690004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.068+0000 7f68d9ffb640 1 -- 172.21.15.88:0/1646373820 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d400a330 con 0x7f68dc14e590 2024-09-21T02:04:07.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.068+0000 7f68e1972640 1 -- 172.21.15.88:0/1646373820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc14e590 msgr2=0x7f68dc0c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.068+0000 7f68e1972640 1 --2- 172.21.15.88:0/1646373820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc14e590 0x7f68dc0c78a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f68dc073900 tx=0x7f68d402a320 comp rx=0 tx=0).stop 2024-09-21T02:04:07.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.069+0000 7f68e1972640 1 -- 172.21.15.88:0/1646373820 shutdown_connections 2024-09-21T02:04:07.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.069+0000 7f68e1972640 1 --2- 172.21.15.88:0/1646373820 >> v2:172.21.15.195:3300/0 conn(0x7f68dc0c2fe0 0x7f68dc15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.069+0000 7f68e1972640 1 --2- 172.21.15.88:0/1646373820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc14e590 0x7f68dc0c78a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.069+0000 7f68e1972640 1 -- 172.21.15.88:0/1646373820 >> 172.21.15.88:0/1646373820 conn(0x7f68dc06d8f0 msgr2=0x7f68dc0ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.069+0000 7f68e1972640 1 -- 172.21.15.88:0/1646373820 shutdown_connections 2024-09-21T02:04:07.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.069+0000 7f68e1972640 1 -- 172.21.15.88:0/1646373820 wait complete. 2024-09-21T02:04:07.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.072+0000 7f68e1972640 1 Processor -- start 2024-09-21T02:04:07.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.072+0000 7f68e1972640 1 -- start start 2024-09-21T02:04:07.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.073+0000 7fd06f0f9640 1 -- 172.21.15.88:0/2671292558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 msgr2=0x7fd0680c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.073+0000 7fd06f0f9640 1 --2- 172.21.15.88:0/2671292558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 0x7fd0680c4480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd06806f6c0 tx=0x7fd06002a980 comp rx=0 tx=0).stop 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.073+0000 7fd06f0f9640 1 -- 172.21.15.88:0/2671292558 shutdown_connections 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.073+0000 7fd06f0f9640 1 --2- 172.21.15.88:0/2671292558 >> v2:172.21.15.195:3300/0 conn(0x7fd068075240 0x7fd068151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.073+0000 7fd06f0f9640 1 --2- 172.21.15.88:0/2671292558 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd068074e30 0x7fd068151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.073+0000 7fd06f0f9640 1 --2- 172.21.15.88:0/2671292558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 0x7fd0680c4480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.073+0000 7fd06f0f9640 1 -- 172.21.15.88:0/2671292558 >> 172.21.15.88:0/2671292558 conn(0x7fd0680ccb90 msgr2=0x7fd0680ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.074+0000 7fd06f0f9640 1 -- 172.21.15.88:0/2671292558 shutdown_connections 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.074+0000 7fd06f0f9640 1 -- 172.21.15.88:0/2671292558 wait complete. 2024-09-21T02:04:07.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7f68e1972640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc0c2fe0 0x7f68dc0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7ffae32ee640 1 -- 172.21.15.88:0/2600283129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40e3530 msgr2=0x7ffad40e3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7ffae32ee640 1 --2- 172.21.15.88:0/2600283129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40e3530 0x7ffad40e3940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffadc000b60 tx=0x7ffadc0a7870 comp rx=0 tx=0).stop 2024-09-21T02:04:07.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7ffae32ee640 1 -- 172.21.15.88:0/2600283129 shutdown_connections 2024-09-21T02:04:07.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7ffae32ee640 1 --2- 172.21.15.88:0/2600283129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffad40e2420 0x7ffad40e2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7ffae32ee640 1 --2- 172.21.15.88:0/2600283129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40e3530 0x7ffad40e3940 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffadc000b60 tx=0x7ffadc0a7870 comp rx=0 tx=0).stop 2024-09-21T02:04:07.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7ffae32ee640 1 --2- 172.21.15.88:0/2600283129 >> v2:172.21.15.195:3300/0 conn(0x7ffad4050710 0x7ffad4007220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7ffae32ee640 1 -- 172.21.15.88:0/2600283129 >> 172.21.15.88:0/2600283129 conn(0x7ffad4050be0 msgr2=0x7ffad4050ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7f68daffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc0c2fe0 0x7f68dc0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7f68daffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc0c2fe0 0x7f68dc0cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52906/0 (socket says 172.21.15.88:52906) 2024-09-21T02:04:07.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7f68daffd640 1 -- 172.21.15.88:0/3060873698 learned_addr learned my addr 172.21.15.88:0/3060873698 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.075+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68dc0c7de0 0x7f68dc149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68dc14e590 0x7f68dc14a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7f68da7fc640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68dc0c7de0 0x7f68dc149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc14e9a0 con 0x7f68dc0c2fe0 2024-09-21T02:04:07.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7faae959a640 1 -- 172.21.15.88:0/3566308917 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400a580 con 0x7faaec073d50 2024-09-21T02:04:07.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7f68db7fe640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68dc14e590 0x7f68dc14a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc158860 con 0x7f68dc0c7de0 2024-09-21T02:04:07.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc1584f0 con 0x7f68dc14e590 2024-09-21T02:04:07.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 -- 172.21.15.88:0/3566308917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec073d50 msgr2=0x7faaec075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 --2- 172.21.15.88:0/3566308917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec073d50 0x7faaec075700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faaec072ba0 tx=0x7faae402b6a0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.076+0000 7ffae32ee640 1 -- 172.21.15.88:0/2600283129 shutdown_connections 2024-09-21T02:04:07.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 -- 172.21.15.88:0/3566308917 shutdown_connections 2024-09-21T02:04:07.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 --2- 172.21.15.88:0/3566308917 >> v2:172.21.15.88:3300/0 conn(0x7faaec154fd0 0x7faaec0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 --2- 172.21.15.88:0/3566308917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec073d50 0x7faaec075700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 -- 172.21.15.88:0/3566308917 >> 172.21.15.88:0/3566308917 conn(0x7faaec06cf70 msgr2=0x7faaec06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f0a0efa8640 1 -- 172.21.15.88:0/1623713792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09f40037a0 msgr2=0x7f09f4003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f68db7fe640 1 -- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68dc0c7de0 msgr2=0x7f68dc149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f68db7fe640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68dc0c7de0 0x7f68dc149e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f68db7fe640 1 -- 172.21.15.88:0/3060873698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc0c2fe0 msgr2=0x7f68dc0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 -- 172.21.15.88:0/3566308917 shutdown_connections 2024-09-21T02:04:07.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7faaf0b20640 1 -- 172.21.15.88:0/3566308917 wait complete. 2024-09-21T02:04:07.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/1623713792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09f40037a0 0x7f09f4003bb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f09f8007dd0 tx=0x7f09f4005830 comp rx=0 tx=0).stop 2024-09-21T02:04:07.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f68db7fe640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc0c2fe0 0x7f68dc0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f68db7fe640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc0ccf40 con 0x7f68dc14e590 2024-09-21T02:04:07.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f6b0be4d640 1 -- 172.21.15.88:0/2710154443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04075770 msgr2=0x7f6b041534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/2710154443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04075770 0x7f6b041534d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b040c7040 tx=0x7f6b0002a5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f68daffd640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc0c2fe0 0x7f68dc0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:04:07.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f0a0efa8640 1 -- 172.21.15.88:0/1623713792 shutdown_connections 2024-09-21T02:04:07.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/1623713792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09f40037a0 0x7f09f4003bb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f0a0efa8640 1 -- 172.21.15.88:0/1623713792 >> 172.21.15.88:0/1623713792 conn(0x7f0a080cda70 msgr2=0x7f0a080cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f68db7fe640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68dc14e590 0x7f68dc14a3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68d000c740 tx=0x7f68d000cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f0a0efa8640 1 -- 172.21.15.88:0/1623713792 shutdown_connections 2024-09-21T02:04:07.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f0a0efa8640 1 -- 172.21.15.88:0/1623713792 wait complete. 2024-09-21T02:04:07.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f68c3fff640 1 -- 172.21.15.88:0/3060873698 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0004460 con 0x7f68dc14e590 2024-09-21T02:04:07.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6b0be4d640 1 -- 172.21.15.88:0/2710154443 shutdown_connections 2024-09-21T02:04:07.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/2710154443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04153a10 0x7f6b04073810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/2710154443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04075770 0x7f6b041534d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6b0be4d640 1 -- 172.21.15.88:0/2710154443 >> 172.21.15.88:0/2710154443 conn(0x7f6b0406d8f0 msgr2=0x7f6b040ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/1374172759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c2e60 msgr2=0x7f6a38152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/1374172759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c2e60 0x7f6a38152900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6a34005fd0 tx=0x7f6a3402c870 comp rx=0 tx=0).stop 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/1374172759 shutdown_connections 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/1374172759 >> v2:172.21.15.195:3300/0 conn(0x7f6a380c7af0 0x7f6a380c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/1374172759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c2e60 0x7f6a38152900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/1374172759 >> 172.21.15.88:0/1374172759 conn(0x7f6a380720b0 msgr2=0x7f6a380cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7fd06f0f9640 1 Processor -- start 2024-09-21T02:04:07.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7fd06f0f9640 1 -- start start 2024-09-21T02:04:07.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f68c3fff640 1 -- 172.21.15.88:0/3060873698 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68d00077a0 con 0x7f68dc14e590 2024-09-21T02:04:07.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f68c3fff640 1 -- 172.21.15.88:0/3060873698 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0007bf0 con 0x7f68dc14e590 2024-09-21T02:04:07.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/1374172759 shutdown_connections 2024-09-21T02:04:07.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/1374172759 wait complete. 2024-09-21T02:04:07.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc1e2e40 con 0x7f68dc14e590 2024-09-21T02:04:07.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.077+0000 7ffae32ee640 1 -- 172.21.15.88:0/2600283129 wait complete. 2024-09-21T02:04:07.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.078+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc1f7350 con 0x7f68dc14e590 2024-09-21T02:04:07.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.079+0000 7f6b0be4d640 1 -- 172.21.15.88:0/2710154443 shutdown_connections 2024-09-21T02:04:07.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.080+0000 7faaf0b20640 1 Processor -- start 2024-09-21T02:04:07.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.080+0000 7faaf0b20640 1 -- start start 2024-09-21T02:04:07.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.079+0000 7f6b0be4d640 1 -- 172.21.15.88:0/2710154443 wait complete. 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.080+0000 7fd06f0f9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 0x7fd06814c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.080+0000 7fd06f0f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd068074e30 0x7fd0680c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7fd06f0f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd068075240 0x7fd06813f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7fd06f0f9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680c7710 con 0x7fd068072bd0 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7fd06f0f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680c7590 con 0x7fd068075240 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7fd06f0f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd0681578b0 con 0x7fd068074e30 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7f0a0efa8640 1 Processor -- start 2024-09-21T02:04:07.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7f0a0efa8640 1 -- start start 2024-09-21T02:04:07.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7f6a3f4d3640 1 Processor -- start 2024-09-21T02:04:07.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7fd06ce95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 0x7fd06814c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.080+0000 7ffae32ee640 1 Processor -- start 2024-09-21T02:04:07.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.080+0000 7ffae32ee640 1 -- start start 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7faaf0b20640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaec073d50 0x7faaec1edb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7f6b0be4d640 1 Processor -- start 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7f6b0be4d640 1 -- start start 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaf0b20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaec0c2ac0 0x7faaec0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaf0b20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec154fd0 0x7faaec148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaf0b20640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0c2640 con 0x7faaec073d50 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaf0b20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0c3fe0 con 0x7faaec154fd0 2024-09-21T02:04:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaf0b20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0c3780 con 0x7faaec0c2ac0 2024-09-21T02:04:07.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7f6a3f4d3640 1 -- start start 2024-09-21T02:04:07.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.081+0000 7fd06ce95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 0x7fd06814c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52912/0 (socket says 172.21.15.88:52912) 2024-09-21T02:04:07.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae32ee640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad4050710 0x7ffad40e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae32ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad40e2420 0x7ffad40db940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaea59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaec073d50 0x7faaec1edb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaea59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaec073d50 0x7faaec1edb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52918/0 (socket says 172.21.15.88:52918) 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faaea59c640 1 -- 172.21.15.88:0/21451312 learned_addr learned my addr 172.21.15.88:0/21451312 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7faae9d9b640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaec0c2ac0 0x7faaec0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae32ee640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffad40d7600 0x7ffad40dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae22ec640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad4050710 0x7ffad40e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae1aeb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad40e2420 0x7ffad40db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae1aeb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad40e2420 0x7ffad40db940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:49988/0 (socket says 172.21.15.88:49988) 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae1aeb640 1 -- 172.21.15.88:0/290538830 learned_addr learned my addr 172.21.15.88:0/290538830 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae2aed640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffad40d7600 0x7ffad40dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7fd06d696640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd068075240 0x7fd06813f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7fd06d696640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd068075240 0x7fd06813f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:49964/0 (socket says 172.21.15.88:49964) 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7fd06d696640 1 -- 172.21.15.88:0/1498875988 learned_addr learned my addr 172.21.15.88:0/1498875988 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7fd067fff640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd068074e30 0x7fd0680c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7fd067fff640 1 -- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd068075240 msgr2=0x7fd06813f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7fd067fff640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd068075240 0x7fd06813f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7fd067fff640 1 -- 172.21.15.88:0/1498875988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 msgr2=0x7fd06814c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7fd067fff640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 0x7fd06814c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7fd067fff640 1 -- 172.21.15.88:0/1498875988 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06814dbb0 con 0x7fd068074e30 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7f68c3fff640 1 -- 172.21.15.88:0/3060873698 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f68d00035f0 con 0x7f68dc14e590 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc005290 con 0x7f68dc14e590 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7f0a0efa8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f40037a0 0x7f0a080c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaead9d640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec154fd0 0x7faaec148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaea59c640 1 -- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaec0c2ac0 msgr2=0x7faaec0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaea59c640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaec0c2ac0 0x7faaec0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaea59c640 1 -- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec154fd0 msgr2=0x7faaec148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaea59c640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec154fd0 0x7faaec148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaea59c640 1 -- 172.21.15.88:0/21451312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec0b0a00 con 0x7faaec073d50 2024-09-21T02:04:07.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaead9d640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec154fd0 0x7faaec148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:07.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffad40e2d70 con 0x7ffad4050710 2024-09-21T02:04:07.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7f0a0efa8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a080705b0 0x7f0a08159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffad40492d0 con 0x7ffad40e2420 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.082+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffad4007760 con 0x7ffad40d7600 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7ffae1aeb640 1 -- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffad40d7600 msgr2=0x7ffad40dbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7ffae1aeb640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffad40d7600 0x7ffad40dbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7ffae1aeb640 1 -- 172.21.15.88:0/290538830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad4050710 msgr2=0x7ffad40e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7ffae1aeb640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad4050710 0x7ffad40e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7ffae1aeb640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad40d6b90 con 0x7ffad40e2420 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7ffae22ec640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad4050710 0x7ffad40e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7ffae1aeb640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad40e2420 0x7ffad40db940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0666e0 tx=0x7ffadc0ae9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7f68c3fff640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68c8078c60 0x7f68c807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.083+0000 7faaea59c640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaec073d50 0x7faaec1edb50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faae4000f80 tx=0x7faae400a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7fd067fff640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd068074e30 0x7fd0680c6090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd05800bfd0 tx=0x7fd05800da70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0cd44640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f40037a0 0x7f0a080c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0cd44640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f40037a0 0x7f0a080c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:35004/0 (socket says 172.21.15.88:35004) 2024-09-21T02:04:07.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f68c3fff640 1 -- 172.21.15.88:0/3060873698 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f68d009b0b0 con 0x7f68dc14e590 2024-09-21T02:04:07.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0cd44640 1 -- 172.21.15.88:0/3580804240 learned_addr learned my addr 172.21.15.88:0/3580804240 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3f4d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a380c2e60 0x7f6a381508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0815d050 0x7f0a0815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6b0be4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b04075770 0x7f6b0414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3f4d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c7af0 0x7f6a381ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3f4d3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a380c91d0 0x7f6a3815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a07fff640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a080705b0 0x7f0a08159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3f4d3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38159190 con 0x7f6a380c91d0 2024-09-21T02:04:07.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3f4d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38159480 con 0x7f6a380c2e60 2024-09-21T02:04:07.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3f4d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3806f1f0 con 0x7f6a380c7af0 2024-09-21T02:04:07.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0a080b5010 con 0x7f0a0815d050 2024-09-21T02:04:07.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0d545640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0815d050 0x7f0a0815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7ffad2ffd640 1 -- 172.21.15.88:0/290538830 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0bd070 con 0x7ffad40e2420 2024-09-21T02:04:07.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7ffad2ffd640 1 -- 172.21.15.88:0/290538830 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffadc0aed20 con 0x7ffad40e2420 2024-09-21T02:04:07.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0a080b5900 con 0x7f0a080705b0 2024-09-21T02:04:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7faadaffd640 1 -- 172.21.15.88:0/21451312 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4040020 con 0x7faaec073d50 2024-09-21T02:04:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0a080ca6d0 con 0x7f09f40037a0 2024-09-21T02:04:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3da70640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a380c91d0 0x7f6a3815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3ca6e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c7af0 0x7f6a381ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3da70640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a380c91d0 0x7f6a3815a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52946/0 (socket says 172.21.15.88:52946) 2024-09-21T02:04:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7f6a3da70640 1 -- 172.21.15.88:0/2156447679 learned_addr learned my addr 172.21.15.88:0/2156447679 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7faadaffd640 1 -- 172.21.15.88:0/21451312 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faae4038040 con 0x7faaec073d50 2024-09-21T02:04:07.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7faadaffd640 1 -- 172.21.15.88:0/21451312 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4037070 con 0x7faaec073d50 2024-09-21T02:04:07.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec1f1530 con 0x7faaec073d50 2024-09-21T02:04:07.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7ffad2ffd640 1 -- 172.21.15.88:0/290538830 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0ad420 con 0x7ffad40e2420 2024-09-21T02:04:07.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad40d7d00 con 0x7ffad40e2420 2024-09-21T02:04:07.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1f1af0 con 0x7faaec073d50 2024-09-21T02:04:07.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.084+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad40d70e0 con 0x7ffad40e2420 2024-09-21T02:04:07.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b0be4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04153a10 0x7f6b0414c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f0a0cd44640 1 -- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a080705b0 msgr2=0x7f0a08159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6a3d26f640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a380c2e60 0x7f6a381508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b0be4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04151a30 0x7f6b04151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b093e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04153a10 0x7f6b0414c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f0a0cd44640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a080705b0 0x7f0a08159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f68daffd640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68c8078c60 0x7f68c807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f0a0cd44640 1 -- 172.21.15.88:0/3580804240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0815d050 msgr2=0x7f0a0815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f0a0cd44640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0815d050 0x7f0a0815d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f0a0cd44640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a081f5b50 con 0x7f09f40037a0 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b093e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04153a10 0x7f6b0414c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:50024/0 (socket says 172.21.15.88:50024) 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b0a3ea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04151a30 0x7f6b04151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b0a3ea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04151a30 0x7f6b04151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.88:35016/0 (socket says 172.21.15.88:35016) 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b093e8640 1 -- 172.21.15.88:0/3891409082 learned_addr learned my addr 172.21.15.88:0/3891409082 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b040c9610 con 0x7f6b04075770 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6b040c9330 con 0x7f6b04153a10 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6b04159780 con 0x7f6b04151a30 2024-09-21T02:04:07.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6a3da70640 1 -- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c7af0 msgr2=0x7f6a381ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6b09be9640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b04075770 0x7f6b0414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6a3da70640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c7af0 0x7f6a381ff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f0a0d545640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0815d050 0x7f0a0815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:04:07.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6a3da70640 1 -- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a380c2e60 msgr2=0x7f6a381508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6a3da70640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a380c2e60 0x7f6a381508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6a3da70640 1 -- 172.21.15.88:0/2156447679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381fe700 con 0x7f6a380c91d0 2024-09-21T02:04:07.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.085+0000 7f6a3da70640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a380c91d0 0x7f6a3815a240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c8c0 tx=0x7f6a3000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f0a07fff640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a080705b0 0x7f0a08159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:07.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6b0a3ea640 1 -- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04153a10 msgr2=0x7f6b0414c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6b0a3ea640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04153a10 0x7f6b0414c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6b0a3ea640 1 -- 172.21.15.88:0/3891409082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b04075770 msgr2=0x7f6b0414e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6b0a3ea640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b04075770 0x7f6b0414e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6b0a3ea640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b04152c90 con 0x7f6b04151a30 2024-09-21T02:04:07.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad4039070 con 0x7ffad40e2420 2024-09-21T02:04:07.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f0a0cd44640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f40037a0 0x7f0a080c79e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a00003ef0 tx=0x7f0a00001780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec06a7f0 con 0x7faaec073d50 2024-09-21T02:04:07.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6b0a3ea640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04151a30 0x7f6b04151e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e670 tx=0x7f6af400eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6a2dffb640 1 -- 172.21.15.88:0/2156447679 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30017070 con 0x7f6a380c91d0 2024-09-21T02:04:07.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a381ff6b0 con 0x7f6a380c91d0 2024-09-21T02:04:07.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f68daffd640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68c8078c60 0x7f68c807b170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f68d4000f80 tx=0x7f68d4004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6a2dffb640 1 -- 172.21.15.88:0/2156447679 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a30005070 con 0x7f6a380c91d0 2024-09-21T02:04:07.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.086+0000 7f6a2dffb640 1 -- 172.21.15.88:0/2156447679 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30005460 con 0x7f6a380c91d0 2024-09-21T02:04:07.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.087+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a380cd210 con 0x7f6a380c91d0 2024-09-21T02:04:07.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.087+0000 7ffad2ffd640 1 -- 172.21.15.88:0/290538830 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffadc0ad070 con 0x7ffad40e2420 2024-09-21T02:04:07.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.087+0000 7ffad2ffd640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffac0078b90 0x7ffac007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.087+0000 7ffad2ffd640 1 -- 172.21.15.88:0/290538830 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffadc1388d0 con 0x7ffad40e2420 2024-09-21T02:04:07.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.088+0000 7ffae22ec640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffac0078b90 0x7ffac007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.088+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a380c8c90 con 0x7f6a380c91d0 2024-09-21T02:04:07.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.088+0000 7ffae22ec640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffac0078b90 0x7ffac007b0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00bf40 tx=0x7ffacc00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.088+0000 7faadaffd640 1 -- 172.21.15.88:0/21451312 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faae4043020 con 0x7faaec073d50 2024-09-21T02:04:07.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f0a057fa640 1 -- 172.21.15.88:0/3580804240 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00015070 con 0x7f09f40037a0 2024-09-21T02:04:07.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7fd0657fa640 1 -- 172.21.15.88:0/1498875988 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058017070 con 0x7fd068074e30 2024-09-21T02:04:07.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f6afa7fc640 1 -- 172.21.15.88:0/3891409082 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4004220 con 0x7f6b04151a30 2024-09-21T02:04:07.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7fd06f0f9640 1 -- 172.21.15.88:0/1498875988 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06814d5f0 con 0x7fd068074e30 2024-09-21T02:04:07.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7fd06f0f9640 1 -- 172.21.15.88:0/1498875988 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0681506d0 con 0x7fd068074e30 2024-09-21T02:04:07.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f68c3fff640 1 -- 172.21.15.88:0/3060873698 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68d0013030 con 0x7f68dc14e590 2024-09-21T02:04:07.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f0a057fa640 1 -- 172.21.15.88:0/3580804240 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a00011900 con 0x7f09f40037a0 2024-09-21T02:04:07.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a080c61b0 con 0x7f09f40037a0 2024-09-21T02:04:07.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f0a057fa640 1 -- 172.21.15.88:0/3580804240 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00011d30 con 0x7f09f40037a0 2024-09-21T02:04:07.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f6afa7fc640 1 -- 172.21.15.88:0/3891409082 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6af400c8b0 con 0x7f6b04151a30 2024-09-21T02:04:07.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f6afa7fc640 1 -- 172.21.15.88:0/3891409082 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af400cd00 con 0x7f6b04151a30 2024-09-21T02:04:07.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b04155550 con 0x7f6b04151a30 2024-09-21T02:04:07.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b04158f00 con 0x7f6b04151a30 2024-09-21T02:04:07.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a081f60f0 con 0x7f09f40037a0 2024-09-21T02:04:07.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7f6a2dffb640 1 -- 172.21.15.88:0/2156447679 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a3001a070 con 0x7f6a380c91d0 2024-09-21T02:04:07.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.090+0000 7f6a2dffb640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6a1c078b90 0x7f6a1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7faadaffd640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faacc078bc0 0x7faacc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.089+0000 7faadaffd640 1 -- 172.21.15.88:0/21451312 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faae40bb4c0 con 0x7faaec073d50 2024-09-21T02:04:07.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.090+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06814c3d0 con 0x7fd068074e30 2024-09-21T02:04:07.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.091+0000 7faae9d9b640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faacc078bc0 0x7faacc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.091+0000 7f6a3d26f640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6a1c078b90 0x7f6a1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.091+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b040c59d0 con 0x7f6b04151a30 2024-09-21T02:04:07.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.091+0000 7fd0657fa640 1 -- 172.21.15.88:0/1498875988 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0580099e0 con 0x7fd068074e30 2024-09-21T02:04:07.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.091+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec005290 con 0x7f09f40037a0 2024-09-21T02:04:07.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.091+0000 7fd0657fa640 1 -- 172.21.15.88:0/1498875988 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058009df0 con 0x7fd068074e30 2024-09-21T02:04:07.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.092+0000 7faae9d9b640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faacc078bc0 0x7faacc07b0d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faaec076a70 tx=0x7faadc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.092+0000 7f6afa7fc640 1 -- 172.21.15.88:0/3891409082 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6af40035f0 con 0x7f6b04151a30 2024-09-21T02:04:07.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.092+0000 7f0a057fa640 1 -- 172.21.15.88:0/3580804240 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a00018070 con 0x7f09f40037a0 2024-09-21T02:04:07.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7f6a3d26f640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6a1c078b90 0x7f6a1c07b0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6a34036000 tx=0x7f6a34004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7ffad2ffd640 1 -- 172.21.15.88:0/290538830 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffadc100620 con 0x7ffad40e2420 2024-09-21T02:04:07.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7f0a057fa640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7f6afa7fc640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6ae8078b90 0x7f6ae807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7f6afa7fc640 1 -- 172.21.15.88:0/3891409082 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6af409b150 con 0x7f6b04151a30 2024-09-21T02:04:07.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7f0a07fff640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7fd0657fa640 1 -- 172.21.15.88:0/1498875988 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd05801a070 con 0x7fd068074e30 2024-09-21T02:04:07.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.093+0000 7f6b09be9640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6ae8078b90 0x7f6ae807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.094+0000 7f0a057fa640 1 -- 172.21.15.88:0/3580804240 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a0009b840 con 0x7f09f40037a0 2024-09-21T02:04:07.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.094+0000 7f6b09be9640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6ae8078b90 0x7f6ae807b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009fd0 tx=0x7f6afc00d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.094+0000 7fd0657fa640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd054078b90 0x7fd05407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.094+0000 7f0a07fff640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f09f8035000 tx=0x7f09f8031040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.094+0000 7fd0657fa640 1 -- 172.21.15.88:0/1498875988 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd05809b160 con 0x7fd068074e30 2024-09-21T02:04:07.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.094+0000 7fd06ce95640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd054078b90 0x7fd05407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.095+0000 7f6a2dffb640 1 -- 172.21.15.88:0/2156447679 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6a30098a00 con 0x7f6a380c91d0 2024-09-21T02:04:07.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.095+0000 7faadaffd640 1 -- 172.21.15.88:0/21451312 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faaec06a7f0 con 0x7faaec073d50 2024-09-21T02:04:07.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.095+0000 7fd06ce95640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd054078b90 0x7fd05407b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd060035000 tx=0x7fd060004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.097+0000 7fd0657fa640 1 -- 172.21.15.88:0/1498875988 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd058062fc0 con 0x7fd068074e30 2024-09-21T02:04:07.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.103+0000 7f6afa7fc640 1 -- 172.21.15.88:0/3891409082 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b040c59d0 con 0x7f6b04151a30 2024-09-21T02:04:07.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.105+0000 7f6a2dffb640 1 -- 172.21.15.88:0/2156447679 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a3009ae20 con 0x7f6a380c91d0 2024-09-21T02:04:07.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.107+0000 7f0a057fa640 1 -- 172.21.15.88:0/3580804240 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a000636b0 con 0x7f09f40037a0 2024-09-21T02:04:07.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.446+0000 7fd6a5a39640 1 -- 172.21.15.88:0/769315780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd684005580 con 0x7fd6a00748b0 2024-09-21T02:04:07.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.447+0000 7fd69d7fa640 1 -- 172.21.15.88:0/769315780 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd688062080 con 0x7fd6a00748b0 2024-09-21T02:04:07.448 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-09-21T02:04:07.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.450+0000 7fd696ffd640 1 -- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd680078c90 msgr2=0x7fd68007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.450+0000 7fd696ffd640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd680078c90 0x7fd68007b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd690038000 tx=0x7fd690004730 comp rx=0 tx=0).stop 2024-09-21T02:04:07.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.450+0000 7fd696ffd640 1 -- 172.21.15.88:0/769315780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00748b0 msgr2=0x7fd6a0167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.450+0000 7fd696ffd640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00748b0 0x7fd6a0167fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c800 tx=0x7fd68800ccc0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 -- 172.21.15.88:0/769315780 shutdown_connections 2024-09-21T02:04:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd680078c90 0x7fd68007b1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a00cd440 0x7fd6a00b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6a00cd030 0x7fd6a0165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 --2- 172.21.15.88:0/769315780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a00748b0 0x7fd6a0167fb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 -- 172.21.15.88:0/769315780 >> 172.21.15.88:0/769315780 conn(0x7fd6a006cf70 msgr2=0x7fd6a016b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 -- 172.21.15.88:0/769315780 shutdown_connections 2024-09-21T02:04:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.451+0000 7fd696ffd640 1 -- 172.21.15.88:0/769315780 wait complete. 2024-09-21T02:04:07.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-09-21T02:04:07.470 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.486+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f68bc005580 con 0x7f68dc14e590 2024-09-21T02:04:07.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.486+0000 7f68c3fff640 1 -- 172.21.15.88:0/3060873698 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f68d0062fa0 con 0x7f68dc14e590 2024-09-21T02:04:07.487 INFO:teuthology.orchestra.run.smithi088.stdout:64424509448 2024-09-21T02:04:07.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.488+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68c8078c60 msgr2=0x7f68c807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.488+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68c8078c60 0x7f68c807b170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f68d4000f80 tx=0x7f68d4004ab0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68dc14e590 msgr2=0x7f68dc14a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68dc14e590 0x7f68dc14a3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68d000c740 tx=0x7f68d000cc00 comp rx=0 tx=0).stop 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 shutdown_connections 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68c8078c60 0x7f68c807b170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68dc14e590 0x7f68dc14a3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68dc0c7de0 0x7f68dc149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 --2- 172.21.15.88:0/3060873698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68dc0c2fe0 0x7f68dc0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 >> 172.21.15.88:0/3060873698 conn(0x7f68dc06d8f0 msgr2=0x7f68dc1e32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 shutdown_connections 2024-09-21T02:04:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.489+0000 7f68e1972640 1 -- 172.21.15.88:0/3060873698 wait complete. 2024-09-21T02:04:07.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.1 2024-09-21T02:04:07.501 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.502+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ffad404e590 con 0x7ffad40e2420 2024-09-21T02:04:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.503+0000 7ffad2ffd640 1 -- 172.21.15.88:0/290538830 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffadc105050 con 0x7ffad40e2420 2024-09-21T02:04:07.504 INFO:teuthology.orchestra.run.smithi088.stdout:64424509449 2024-09-21T02:04:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffac0078b90 msgr2=0x7ffac007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffac0078b90 0x7ffac007b0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00bf40 tx=0x7ffacc00b2e0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad40e2420 msgr2=0x7ffad40db940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad40e2420 0x7ffad40db940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0666e0 tx=0x7ffadc0ae9a0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 shutdown_connections 2024-09-21T02:04:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffac0078b90 0x7ffac007b0a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffad40d7600 0x7ffad40dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffad40e2420 0x7ffad40db940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 --2- 172.21.15.88:0/290538830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad4050710 0x7ffad40e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 >> 172.21.15.88:0/290538830 conn(0x7ffad4050be0 msgr2=0x7ffad404efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 shutdown_connections 2024-09-21T02:04:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.507+0000 7ffae32ee640 1 -- 172.21.15.88:0/290538830 wait complete. 2024-09-21T02:04:07.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.516+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6a380cccc0 con 0x7f6a380c91d0 2024-09-21T02:04:07.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.516+0000 7f6a2dffb640 1 -- 172.21.15.88:0/2156447679 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a380c42e0 con 0x7f6a380c91d0 2024-09-21T02:04:07.517 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2024-09-21T02:04:07.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509449 for osd.3 2024-09-21T02:04:07.520 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6a1c078b90 msgr2=0x7f6a1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6a1c078b90 0x7f6a1c07b0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6a34036000 tx=0x7f6a34004730 comp rx=0 tx=0).stop 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a380c91d0 msgr2=0x7f6a3815a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a380c91d0 0x7f6a3815a240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c8c0 tx=0x7f6a3000cd80 comp rx=0 tx=0).stop 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 shutdown_connections 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6a1c078b90 0x7f6a1c07b0a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a380c91d0 0x7f6a3815a240 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a380c7af0 0x7f6a381ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 --2- 172.21.15.88:0/2156447679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a380c2e60 0x7f6a381508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.518+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 >> 172.21.15.88:0/2156447679 conn(0x7f6a380720b0 msgr2=0x7f6a380ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.519+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 shutdown_connections 2024-09-21T02:04:07.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.519+0000 7f6a3f4d3640 1 -- 172.21.15.88:0/2156447679 wait complete. 2024-09-21T02:04:07.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.521+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7faaec077550 con 0x7faaec073d50 2024-09-21T02:04:07.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.522+0000 7faadaffd640 1 -- 172.21.15.88:0/21451312 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faae404a030 con 0x7faaec073d50 2024-09-21T02:04:07.523 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2024-09-21T02:04:07.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.522+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd06815bf60 con 0x7fd068074e30 2024-09-21T02:04:07.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.523+0000 7fd0657fa640 1 -- 172.21.15.88:0/1498875988 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0580679f0 con 0x7fd068074e30 2024-09-21T02:04:07.524 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T02:04:07.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faacc078bc0 msgr2=0x7faacc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faacc078bc0 0x7faacc07b0d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faaec076a70 tx=0x7faadc009040 comp rx=0 tx=0).stop 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaec073d50 msgr2=0x7faaec1edb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaec073d50 0x7faaec1edb50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faae4000f80 tx=0x7faae400a490 comp rx=0 tx=0).stop 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 shutdown_connections 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faacc078bc0 0x7faacc07b0d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaec154fd0 0x7faaec148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaec0c2ac0 0x7faaec0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 --2- 172.21.15.88:0/21451312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaec073d50 0x7faaec1edb50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 >> 172.21.15.88:0/21451312 conn(0x7faaec06cf70 msgr2=0x7faaec0ca260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 shutdown_connections 2024-09-21T02:04:07.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.524+0000 7faaf0b20640 1 -- 172.21.15.88:0/21451312 wait complete. 2024-09-21T02:04:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd054078b90 msgr2=0x7fd05407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd054078b90 0x7fd05407b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd060035000 tx=0x7fd060004ab0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd068074e30 msgr2=0x7fd0680c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd068074e30 0x7fd0680c6090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd05800bfd0 tx=0x7fd05800da70 comp rx=0 tx=0).stop 2024-09-21T02:04:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 shutdown_connections 2024-09-21T02:04:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd054078b90 0x7fd05407b0a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd068075240 0x7fd06813f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd068074e30 0x7fd0680c6090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 --2- 172.21.15.88:0/1498875988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd068072bd0 0x7fd06814c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 >> 172.21.15.88:0/1498875988 conn(0x7fd0680ccb90 msgr2=0x7fd068154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 shutdown_connections 2024-09-21T02:04:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.525+0000 7fd04effd640 1 -- 172.21.15.88:0/1498875988 wait complete. 2024-09-21T02:04:07.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-09-21T02:04:07.532 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.533+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6b0406e4b0 con 0x7f6b04151a30 2024-09-21T02:04:07.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.534+0000 7f6afa7fc640 1 -- 172.21.15.88:0/3891409082 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6af4062fc0 con 0x7f6b04151a30 2024-09-21T02:04:07.534 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T02:04:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6ae8078b90 msgr2=0x7f6ae807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6ae8078b90 0x7f6ae807b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009fd0 tx=0x7f6afc00d040 comp rx=0 tx=0).stop 2024-09-21T02:04:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04151a30 msgr2=0x7f6b04151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04151a30 0x7f6b04151e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6af400e670 tx=0x7f6af400eb30 comp rx=0 tx=0).stop 2024-09-21T02:04:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 shutdown_connections 2024-09-21T02:04:07.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6ae8078b90 0x7f6ae807b0a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b04151a30 0x7f6b04151e40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b04153a10 0x7f6b0414c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 --2- 172.21.15.88:0/3891409082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b04075770 0x7f6b0414e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 >> 172.21.15.88:0/3891409082 conn(0x7f6b0406d8f0 msgr2=0x7f6b040cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 shutdown_connections 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.537+0000 7f6b0be4d640 1 -- 172.21.15.88:0/3891409082 wait complete. 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.538+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f09ec005580 con 0x7f09f40037a0 2024-09-21T02:04:07.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.538+0000 7f0a057fa640 1 -- 172.21.15.88:0/3580804240 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0a000680e0 con 0x7f09f40037a0 2024-09-21T02:04:07.540 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.540+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 msgr2=0x7f09e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.540+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f09f8035000 tx=0x7f09f8031040 comp rx=0 tx=0).stop 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.540+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f40037a0 msgr2=0x7f0a080c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.540+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f40037a0 0x7f0a080c79e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a00003ef0 tx=0x7f0a00001780 comp rx=0 tx=0).stop 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 shutdown_connections 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f09e8078bc0 0x7f09e807b0d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0815d050 0x7f0a0815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a080705b0 0x7f0a08159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 --2- 172.21.15.88:0/3580804240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09f40037a0 0x7f0a080c79e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 >> 172.21.15.88:0/3580804240 conn(0x7f0a080cda70 msgr2=0x7f0a0815fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 shutdown_connections 2024-09-21T02:04:07.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.541+0000 7f0a0efa8640 1 -- 172.21.15.88:0/3580804240 wait complete. 2024-09-21T02:04:07.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.5 2024-09-21T02:04:07.544 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.2 2024-09-21T02:04:07.544 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.7 2024-09-21T02:04:07.551 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-09-21T02:04:07.554 DEBUG:teuthology.parallel:result is None 2024-09-21T02:04:07.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:04:07.554 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.706+0000 7fc09024d640 1 -- 172.21.15.88:0/3962757275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc074005330 msgr2=0x7fc07400b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.706+0000 7fc09024d640 1 --2- 172.21.15.88:0/3962757275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc074005330 0x7fc07400b9e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc080001d90 tx=0x7fc074005830 comp rx=0 tx=0).stop 2024-09-21T02:04:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.706+0000 7fc09024d640 1 -- 172.21.15.88:0/3962757275 shutdown_connections 2024-09-21T02:04:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.706+0000 7fc09024d640 1 --2- 172.21.15.88:0/3962757275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc074005330 0x7fc07400b9e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.706+0000 7fc09024d640 1 -- 172.21.15.88:0/3962757275 >> 172.21.15.88:0/3962757275 conn(0x7fc0880cd9f0 msgr2=0x7fc0880cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.706+0000 7fc09024d640 1 -- 172.21.15.88:0/3962757275 shutdown_connections 2024-09-21T02:04:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.706+0000 7fc09024d640 1 -- 172.21.15.88:0/3962757275 wait complete. 2024-09-21T02:04:07.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.707+0000 7fc09024d640 1 Processor -- start 2024-09-21T02:04:07.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.707+0000 7fc09024d640 1 -- start start 2024-09-21T02:04:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc074005330 0x7fc08815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0880c43f0 0x7fc0881e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0880d05c0 0x7fc088149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc08815fd80 con 0x7fc0880d05c0 2024-09-21T02:04:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc08815c640 con 0x7fc074005330 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc08806dcd0 con 0x7fc0880c43f0 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0880d05c0 0x7fc088149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0880d05c0 0x7fc088149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52988/0 (socket says 172.21.15.88:52988) 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 -- 172.21.15.88:0/2674537009 learned_addr learned my addr 172.21.15.88:0/2674537009 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08dfe9640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc074005330 0x7fc08815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08d7e8640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0880c43f0 0x7fc0881e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 -- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0880c43f0 msgr2=0x7fc0881e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0880c43f0 0x7fc0881e4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 -- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc074005330 msgr2=0x7fc08815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc074005330 0x7fc08815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 -- 172.21.15.88:0/2674537009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0881e53e0 con 0x7fc0880d05c0 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08dfe9640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc074005330 0x7fc08815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc08e7ea640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0880d05c0 0x7fc088149170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc080001d90 tx=0x7fc080006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc07e7fc640 1 -- 172.21.15.88:0/2674537009 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080003d80 con 0x7fc0880d05c0 2024-09-21T02:04:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc07e7fc640 1 -- 172.21.15.88:0/2674537009 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc080005ba0 con 0x7fc0880d05c0 2024-09-21T02:04:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc07e7fc640 1 -- 172.21.15.88:0/2674537009 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080005ff0 con 0x7fc0880d05c0 2024-09-21T02:04:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08814da50 con 0x7fc0880d05c0 2024-09-21T02:04:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.709+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08815e300 con 0x7fc0880d05c0 2024-09-21T02:04:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.710+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0881e7010 con 0x7fc0880d05c0 2024-09-21T02:04:07.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.711+0000 7fc07e7fc640 1 -- 172.21.15.88:0/2674537009 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc080005030 con 0x7fc0880d05c0 2024-09-21T02:04:07.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.712+0000 7fc07e7fc640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc06c078b90 0x7fc06c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:07.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.712+0000 7fc07e7fc640 1 -- 172.21.15.88:0/2674537009 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc0800bab50 con 0x7fc0880d05c0 2024-09-21T02:04:07.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.712+0000 7fc08dfe9640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc06c078b90 0x7fc06c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:07.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.712+0000 7fc08dfe9640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc06c078b90 0x7fc06c07b0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc084001220 tx=0x7fc0840067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:07.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.716+0000 7fc07e7fc640 1 -- 172.21.15.88:0/2674537009 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc080043060 con 0x7fc0880d05c0 2024-09-21T02:04:07.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.889+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc08806cd60 con 0x7fc06c078b90 2024-09-21T02:04:07.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.894+0000 7fc07e7fc640 1 -- 172.21.15.88:0/2674537009 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251824 (secure 0 0 0) 0x7fc08806cd60 con 0x7fc06c078b90 2024-09-21T02:04:07.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:07.898 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc06c078b90 msgr2=0x7fc06c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc06c078b90 0x7fc06c07b0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc084001220 tx=0x7fc0840067a0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0880d05c0 msgr2=0x7fc088149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0880d05c0 0x7fc088149170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc080001d90 tx=0x7fc080006d00 comp rx=0 tx=0).stop 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 shutdown_connections 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc06c078b90 0x7fc06c07b0a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0880d05c0 0x7fc088149170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0880c43f0 0x7fc0881e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 --2- 172.21.15.88:0/2674537009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc074005330 0x7fc08815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 >> 172.21.15.88:0/2674537009 conn(0x7fc0880cd9f0 msgr2=0x7fc088155660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 shutdown_connections 2024-09-21T02:04:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:07.897+0000 7fc09024d640 1 -- 172.21.15.88:0/2674537009 wait complete. 2024-09-21T02:04:07.909 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-21T02:04:06.820090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239848,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734872,"statfs":{"total":773094113280,"available":772848508928,"internally_reserved":0,"allocated":2924544,"data_stored":1767532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.759880"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074727+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:03:59.074727+0000","last_peered":"2024-09-21T02:03:59.074727+0000","last_clean":"2024-09-21T02:03:59.074727+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:03:59.074727+0000","last_undegraded":"2024-09-21T02:03:59.074727+0000","last_fullsized":"2024-09-21T02:03:59.074727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073213+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:03:59.073213+0000","last_peered":"2024-09-21T02:03:59.073213+0000","last_clean":"2024-09-21T02:03:59.073213+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:03:59.073213+0000","last_undegraded":"2024-09-21T02:03:59.073213+0000","last_fullsized":"2024-09-21T02:03:59.073213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836539+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:03:59.836539+0000","last_peered":"2024-09-21T02:03:59.836539+0000","last_clean":"2024-09-21T02:03:59.836539+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:03:59.836539+0000","last_undegraded":"2024-09-21T02:03:59.836539+0000","last_fullsized":"2024-09-21T02:03:59.836539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075220+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:03:59.075220+0000","last_peered":"2024-09-21T02:03:59.075220+0000","last_clean":"2024-09-21T02:03:59.075220+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:03:59.075220+0000","last_undegraded":"2024-09-21T02:03:59.075220+0000","last_fullsized":"2024-09-21T02:03:59.075220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074990+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:03:59.074990+0000","last_peered":"2024-09-21T02:03:59.074990+0000","last_clean":"2024-09-21T02:03:59.074990+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:03:59.074990+0000","last_undegraded":"2024-09-21T02:03:59.074990+0000","last_fullsized":"2024-09-21T02:03:59.074990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074223+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:03:59.074223+0000","last_peered":"2024-09-21T02:03:59.074223+0000","last_clean":"2024-09-21T02:03:59.074223+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:03:59.074223+0000","last_undegraded":"2024-09-21T02:03:59.074223+0000","last_fullsized":"2024-09-21T02:03:59.074223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074095+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:03:59.074095+0000","last_peered":"2024-09-21T02:03:59.074095+0000","last_clean":"2024-09-21T02:03:59.074095+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:03:59.074095+0000","last_undegraded":"2024-09-21T02:03:59.074095+0000","last_fullsized":"2024-09-21T02:03:59.074095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835347+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:03:59.835347+0000","last_peered":"2024-09-21T02:03:59.835347+0000","last_clean":"2024-09-21T02:03:59.835347+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:03:59.835347+0000","last_undegraded":"2024-09-21T02:03:59.835347+0000","last_fullsized":"2024-09-21T02:03:59.835347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074637+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:03:59.074637+0000","last_peered":"2024-09-21T02:03:59.074637+0000","last_clean":"2024-09-21T02:03:59.074637+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:03:59.074637+0000","last_undegraded":"2024-09-21T02:03:59.074637+0000","last_fullsized":"2024-09-21T02:03:59.074637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835966+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:03:59.835966+0000","last_peered":"2024-09-21T02:03:59.835966+0000","last_clean":"2024-09-21T02:03:59.835966+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:03:59.835966+0000","last_undegraded":"2024-09-21T02:03:59.835966+0000","last_fullsized":"2024-09-21T02:03:59.835966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835587+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:03:59.835587+0000","last_peered":"2024-09-21T02:03:59.835587+0000","last_clean":"2024-09-21T02:03:59.835587+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:03:59.835587+0000","last_undegraded":"2024-09-21T02:03:59.835587+0000","last_fullsized":"2024-09-21T02:03:59.835587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781637+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:03:59.781637+0000","last_peered":"2024-09-21T02:03:59.781637+0000","last_clean":"2024-09-21T02:03:59.781637+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:03:59.781637+0000","last_undegraded":"2024-09-21T02:03:59.781637+0000","last_fullsized":"2024-09-21T02:03:59.781637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835918+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:03:59.835918+0000","last_peered":"2024-09-21T02:03:59.835918+0000","last_clean":"2024-09-21T02:03:59.835918+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:03:59.835918+0000","last_undegraded":"2024-09-21T02:03:59.835918+0000","last_fullsized":"2024-09-21T02:03:59.835918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836186+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:03:59.836186+0000","last_peered":"2024-09-21T02:03:59.836186+0000","last_clean":"2024-09-21T02:03:59.836186+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:03:59.836186+0000","last_undegraded":"2024-09-21T02:03:59.836186+0000","last_fullsized":"2024-09-21T02:03:59.836186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072763+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:03:59.072763+0000","last_peered":"2024-09-21T02:03:59.072763+0000","last_clean":"2024-09-21T02:03:59.072763+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:03:59.072763+0000","last_undegraded":"2024-09-21T02:03:59.072763+0000","last_fullsized":"2024-09-21T02:03:59.072763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075548+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:03:59.075548+0000","last_peered":"2024-09-21T02:03:59.075548+0000","last_clean":"2024-09-21T02:03:59.075548+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:03:59.075548+0000","last_undegraded":"2024-09-21T02:03:59.075548+0000","last_fullsized":"2024-09-21T02:03:59.075548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075910+0000","last_change":"2024-09-21T02:03:58.074570+0000","last_active":"2024-09-21T02:03:59.075910+0000","last_peered":"2024-09-21T02:03:59.075910+0000","last_clean":"2024-09-21T02:03:59.075910+0000","last_became_active":"2024-09-21T02:03:58.074271+0000","last_became_peered":"2024-09-21T02:03:58.074271+0000","last_unstale":"2024-09-21T02:03:59.075910+0000","last_undegraded":"2024-09-21T02:03:59.075910+0000","last_fullsized":"2024-09-21T02:03:59.075910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.292792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835423+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:03:59.835423+0000","last_peered":"2024-09-21T02:03:59.835423+0000","last_clean":"2024-09-21T02:03:59.835423+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:03:59.835423+0000","last_undegraded":"2024-09-21T02:03:59.835423+0000","last_fullsized":"2024-09-21T02:03:59.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076132+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:03:59.076132+0000","last_peered":"2024-09-21T02:03:59.076132+0000","last_clean":"2024-09-21T02:03:59.076132+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:03:59.076132+0000","last_undegraded":"2024-09-21T02:03:59.076132+0000","last_fullsized":"2024-09-21T02:03:59.076132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835510+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:03:59.835510+0000","last_peered":"2024-09-21T02:03:59.835510+0000","last_clean":"2024-09-21T02:03:59.835510+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:03:59.835510+0000","last_undegraded":"2024-09-21T02:03:59.835510+0000","last_fullsized":"2024-09-21T02:03:59.835510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836526+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:03:59.836526+0000","last_peered":"2024-09-21T02:03:59.836526+0000","last_clean":"2024-09-21T02:03:59.836526+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:03:59.836526+0000","last_undegraded":"2024-09-21T02:03:59.836526+0000","last_fullsized":"2024-09-21T02:03:59.836526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075251+0000","last_change":"2024-09-21T02:03:58.076378+0000","last_active":"2024-09-21T02:03:59.075251+0000","last_peered":"2024-09-21T02:03:59.075251+0000","last_clean":"2024-09-21T02:03:59.075251+0000","last_became_active":"2024-09-21T02:03:58.076075+0000","last_became_peered":"2024-09-21T02:03:58.076075+0000","last_unstale":"2024-09-21T02:03:59.075251+0000","last_undegraded":"2024-09-21T02:03:59.075251+0000","last_fullsized":"2024-09-21T02:03:59.075251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:54.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835596+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:03:59.835596+0000","last_peered":"2024-09-21T02:03:59.835596+0000","last_clean":"2024-09-21T02:03:59.835596+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:03:59.835596+0000","last_undegraded":"2024-09-21T02:03:59.835596+0000","last_fullsized":"2024-09-21T02:03:59.835596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074571+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:03:59.074571+0000","last_peered":"2024-09-21T02:03:59.074571+0000","last_clean":"2024-09-21T02:03:59.074571+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:03:59.074571+0000","last_undegraded":"2024-09-21T02:03:59.074571+0000","last_fullsized":"2024-09-21T02:03:59.074571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836762+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:03:59.836762+0000","last_peered":"2024-09-21T02:03:59.836762+0000","last_clean":"2024-09-21T02:03:59.836762+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:03:59.836762+0000","last_undegraded":"2024-09-21T02:03:59.836762+0000","last_fullsized":"2024-09-21T02:03:59.836762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835955+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:03:59.835955+0000","last_peered":"2024-09-21T02:03:59.835955+0000","last_clean":"2024-09-21T02:03:59.835955+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:03:59.835955+0000","last_undegraded":"2024-09-21T02:03:59.835955+0000","last_fullsized":"2024-09-21T02:03:59.835955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073053+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:03:59.073053+0000","last_peered":"2024-09-21T02:03:59.073053+0000","last_clean":"2024-09-21T02:03:59.073053+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:03:59.073053+0000","last_undegraded":"2024-09-21T02:03:59.073053+0000","last_fullsized":"2024-09-21T02:03:59.073053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836383+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:03:59.836383+0000","last_peered":"2024-09-21T02:03:59.836383+0000","last_clean":"2024-09-21T02:03:59.836383+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:03:59.836383+0000","last_undegraded":"2024-09-21T02:03:59.836383+0000","last_fullsized":"2024-09-21T02:03:59.836383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836861+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:03:59.836861+0000","last_peered":"2024-09-21T02:03:59.836861+0000","last_clean":"2024-09-21T02:03:59.836861+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:03:59.836861+0000","last_undegraded":"2024-09-21T02:03:59.836861+0000","last_fullsized":"2024-09-21T02:03:59.836861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075554+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:03:59.075554+0000","last_peered":"2024-09-21T02:03:59.075554+0000","last_clean":"2024-09-21T02:03:59.075554+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:03:59.075554+0000","last_undegraded":"2024-09-21T02:03:59.075554+0000","last_fullsized":"2024-09-21T02:03:59.075554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835391+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:03:59.835391+0000","last_peered":"2024-09-21T02:03:59.835391+0000","last_clean":"2024-09-21T02:03:59.835391+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:03:59.835391+0000","last_undegraded":"2024-09-21T02:03:59.835391+0000","last_fullsized":"2024-09-21T02:03:59.835391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835439+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:03:59.835439+0000","last_peered":"2024-09-21T02:03:59.835439+0000","last_clean":"2024-09-21T02:03:59.835439+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:03:59.835439+0000","last_undegraded":"2024-09-21T02:03:59.835439+0000","last_fullsized":"2024-09-21T02:03:59.835439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781525+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:03:59.781525+0000","last_peered":"2024-09-21T02:03:59.781525+0000","last_clean":"2024-09-21T02:03:59.781525+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:03:59.781525+0000","last_undegraded":"2024-09-21T02:03:59.781525+0000","last_fullsized":"2024-09-21T02:03:59.781525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836732+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:03:59.836732+0000","last_peered":"2024-09-21T02:03:59.836732+0000","last_clean":"2024-09-21T02:03:59.836732+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:03:59.836732+0000","last_undegraded":"2024-09-21T02:03:59.836732+0000","last_fullsized":"2024-09-21T02:03:59.836732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836856+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:03:59.836856+0000","last_peered":"2024-09-21T02:03:59.836856+0000","last_clean":"2024-09-21T02:03:59.836856+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:03:59.836856+0000","last_undegraded":"2024-09-21T02:03:59.836856+0000","last_fullsized":"2024-09-21T02:03:59.836856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835620+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:03:59.835620+0000","last_peered":"2024-09-21T02:03:59.835620+0000","last_clean":"2024-09-21T02:03:59.835620+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:03:59.835620+0000","last_undegraded":"2024-09-21T02:03:59.835620+0000","last_fullsized":"2024-09-21T02:03:59.835620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835432+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:03:59.835432+0000","last_peered":"2024-09-21T02:03:59.835432+0000","last_clean":"2024-09-21T02:03:59.835432+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:03:59.835432+0000","last_undegraded":"2024-09-21T02:03:59.835432+0000","last_fullsized":"2024-09-21T02:03:59.835432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835573+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:03:59.835573+0000","last_peered":"2024-09-21T02:03:59.835573+0000","last_clean":"2024-09-21T02:03:59.835573+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:03:59.835573+0000","last_undegraded":"2024-09-21T02:03:59.835573+0000","last_fullsized":"2024-09-21T02:03:59.835573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074657+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:03:59.074657+0000","last_peered":"2024-09-21T02:03:59.074657+0000","last_clean":"2024-09-21T02:03:59.074657+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:03:59.074657+0000","last_undegraded":"2024-09-21T02:03:59.074657+0000","last_fullsized":"2024-09-21T02:03:59.074657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072701+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:03:59.072701+0000","last_peered":"2024-09-21T02:03:59.072701+0000","last_clean":"2024-09-21T02:03:59.072701+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:03:59.072701+0000","last_undegraded":"2024-09-21T02:03:59.072701+0000","last_fullsized":"2024-09-21T02:03:59.072701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835536+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:03:59.835536+0000","last_peered":"2024-09-21T02:03:59.835536+0000","last_clean":"2024-09-21T02:03:59.835536+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:03:59.835536+0000","last_undegraded":"2024-09-21T02:03:59.835536+0000","last_fullsized":"2024-09-21T02:03:59.835536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074748+0000","last_change":"2024-09-21T02:03:58.075434+0000","last_active":"2024-09-21T02:03:59.074748+0000","last_peered":"2024-09-21T02:03:59.074748+0000","last_clean":"2024-09-21T02:03:59.074748+0000","last_became_active":"2024-09-21T02:03:58.075130+0000","last_became_peered":"2024-09-21T02:03:58.075130+0000","last_unstale":"2024-09-21T02:03:59.074748+0000","last_undegraded":"2024-09-21T02:03:59.074748+0000","last_fullsized":"2024-09-21T02:03:59.074748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:56.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074818+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:03:59.074818+0000","last_peered":"2024-09-21T02:03:59.074818+0000","last_clean":"2024-09-21T02:03:59.074818+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:03:59.074818+0000","last_undegraded":"2024-09-21T02:03:59.074818+0000","last_fullsized":"2024-09-21T02:03:59.074818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836482+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:03:59.836482+0000","last_peered":"2024-09-21T02:03:59.836482+0000","last_clean":"2024-09-21T02:03:59.836482+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:03:59.836482+0000","last_undegraded":"2024-09-21T02:03:59.836482+0000","last_fullsized":"2024-09-21T02:03:59.836482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069302+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:04:03.069302+0000","last_peered":"2024-09-21T02:04:03.069302+0000","last_clean":"2024-09-21T02:04:03.069302+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:04:03.069302+0000","last_undegraded":"2024-09-21T02:04:03.069302+0000","last_fullsized":"2024-09-21T02:04:03.069302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781695+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:03:59.781695+0000","last_peered":"2024-09-21T02:03:59.781695+0000","last_clean":"2024-09-21T02:03:59.781695+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:03:59.781695+0000","last_undegraded":"2024-09-21T02:03:59.781695+0000","last_fullsized":"2024-09-21T02:03:59.781695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.071755+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:04:03.071755+0000","last_peered":"2024-09-21T02:04:03.071755+0000","last_clean":"2024-09-21T02:04:03.071755+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:04:03.071755+0000","last_undegraded":"2024-09-21T02:04:03.071755+0000","last_fullsized":"2024-09-21T02:04:03.071755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075568+0000","last_change":"2024-09-21T02:03:58.080947+0000","last_active":"2024-09-21T02:03:59.075568+0000","last_peered":"2024-09-21T02:03:59.075568+0000","last_clean":"2024-09-21T02:03:59.075568+0000","last_became_active":"2024-09-21T02:03:58.080632+0000","last_became_peered":"2024-09-21T02:03:58.080632+0000","last_unstale":"2024-09-21T02:03:59.075568+0000","last_undegraded":"2024-09-21T02:03:59.075568+0000","last_fullsized":"2024-09-21T02:03:59.075568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:40.307689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070474+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:03.070474+0000","last_peered":"2024-09-21T02:04:03.070474+0000","last_clean":"2024-09-21T02:04:03.070474+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:03.070474+0000","last_undegraded":"2024-09-21T02:04:03.070474+0000","last_fullsized":"2024-09-21T02:04:03.070474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836343+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:03:59.836343+0000","last_peered":"2024-09-21T02:03:59.836343+0000","last_clean":"2024-09-21T02:03:59.836343+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:03:59.836343+0000","last_undegraded":"2024-09-21T02:03:59.836343+0000","last_fullsized":"2024-09-21T02:03:59.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075336+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:03:59.075336+0000","last_peered":"2024-09-21T02:03:59.075336+0000","last_clean":"2024-09-21T02:03:59.075336+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:03:59.075336+0000","last_undegraded":"2024-09-21T02:03:59.075336+0000","last_fullsized":"2024-09-21T02:03:59.075336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075210+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:03:59.075210+0000","last_peered":"2024-09-21T02:03:59.075210+0000","last_clean":"2024-09-21T02:03:59.075210+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:03:59.075210+0000","last_undegraded":"2024-09-21T02:03:59.075210+0000","last_fullsized":"2024-09-21T02:03:59.075210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835374+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:03:59.835374+0000","last_peered":"2024-09-21T02:03:59.835374+0000","last_clean":"2024-09-21T02:03:59.835374+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:03:59.835374+0000","last_undegraded":"2024-09-21T02:03:59.835374+0000","last_fullsized":"2024-09-21T02:03:59.835374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835641+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:03:59.835641+0000","last_peered":"2024-09-21T02:03:59.835641+0000","last_clean":"2024-09-21T02:03:59.835641+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:03:59.835641+0000","last_undegraded":"2024-09-21T02:03:59.835641+0000","last_fullsized":"2024-09-21T02:03:59.835641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.072054+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:04:03.072054+0000","last_peered":"2024-09-21T02:04:03.072054+0000","last_clean":"2024-09-21T02:04:03.072054+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:04:03.072054+0000","last_undegraded":"2024-09-21T02:04:03.072054+0000","last_fullsized":"2024-09-21T02:04:03.072054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074582+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:03:59.074582+0000","last_peered":"2024-09-21T02:03:59.074582+0000","last_clean":"2024-09-21T02:03:59.074582+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:03:59.074582+0000","last_undegraded":"2024-09-21T02:03:59.074582+0000","last_fullsized":"2024-09-21T02:03:59.074582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781594+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:03:59.781594+0000","last_peered":"2024-09-21T02:03:59.781594+0000","last_clean":"2024-09-21T02:03:59.781594+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:03:59.781594+0000","last_undegraded":"2024-09-21T02:03:59.781594+0000","last_fullsized":"2024-09-21T02:03:59.781594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074555+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:03:59.074555+0000","last_peered":"2024-09-21T02:03:59.074555+0000","last_clean":"2024-09-21T02:03:59.074555+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:03:59.074555+0000","last_undegraded":"2024-09-21T02:03:59.074555+0000","last_fullsized":"2024-09-21T02:03:59.074555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.071532+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:03.071532+0000","last_peered":"2024-09-21T02:04:03.071532+0000","last_clean":"2024-09-21T02:04:03.071532+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:03.071532+0000","last_undegraded":"2024-09-21T02:04:03.071532+0000","last_fullsized":"2024-09-21T02:04:03.071532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074911+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:03:59.074911+0000","last_peered":"2024-09-21T02:03:59.074911+0000","last_clean":"2024-09-21T02:03:59.074911+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:03:59.074911+0000","last_undegraded":"2024-09-21T02:03:59.074911+0000","last_fullsized":"2024-09-21T02:03:59.074911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073109+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:03:59.073109+0000","last_peered":"2024-09-21T02:03:59.073109+0000","last_clean":"2024-09-21T02:03:59.073109+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:03:59.073109+0000","last_undegraded":"2024-09-21T02:03:59.073109+0000","last_fullsized":"2024-09-21T02:03:59.073109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075681+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:03:59.075681+0000","last_peered":"2024-09-21T02:03:59.075681+0000","last_clean":"2024-09-21T02:03:59.075681+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:03:59.075681+0000","last_undegraded":"2024-09-21T02:03:59.075681+0000","last_fullsized":"2024-09-21T02:03:59.075681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835624+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:03:59.835624+0000","last_peered":"2024-09-21T02:03:59.835624+0000","last_clean":"2024-09-21T02:03:59.835624+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:03:59.835624+0000","last_undegraded":"2024-09-21T02:03:59.835624+0000","last_fullsized":"2024-09-21T02:03:59.835624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075817+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:03:59.075817+0000","last_peered":"2024-09-21T02:03:59.075817+0000","last_clean":"2024-09-21T02:03:59.075817+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:03:59.075817+0000","last_undegraded":"2024-09-21T02:03:59.075817+0000","last_fullsized":"2024-09-21T02:03:59.075817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074041+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:03:59.074041+0000","last_peered":"2024-09-21T02:03:59.074041+0000","last_clean":"2024-09-21T02:03:59.074041+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:03:59.074041+0000","last_undegraded":"2024-09-21T02:03:59.074041+0000","last_fullsized":"2024-09-21T02:03:59.074041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074376+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:03:59.074376+0000","last_peered":"2024-09-21T02:03:59.074376+0000","last_clean":"2024-09-21T02:03:59.074376+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:03:59.074376+0000","last_undegraded":"2024-09-21T02:03:59.074376+0000","last_fullsized":"2024-09-21T02:03:59.074376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835923+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:03:59.835923+0000","last_peered":"2024-09-21T02:03:59.835923+0000","last_clean":"2024-09-21T02:03:59.835923+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:03:59.835923+0000","last_undegraded":"2024-09-21T02:03:59.835923+0000","last_fullsized":"2024-09-21T02:03:59.835923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073892+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:03:59.073892+0000","last_peered":"2024-09-21T02:03:59.073892+0000","last_clean":"2024-09-21T02:03:59.073892+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:03:59.073892+0000","last_undegraded":"2024-09-21T02:03:59.073892+0000","last_fullsized":"2024-09-21T02:03:59.073892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835986+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:03:59.835986+0000","last_peered":"2024-09-21T02:03:59.835986+0000","last_clean":"2024-09-21T02:03:59.835986+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:03:59.835986+0000","last_undegraded":"2024-09-21T02:03:59.835986+0000","last_fullsized":"2024-09-21T02:03:59.835986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:05.466556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835266+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:03:59.835266+0000","last_peered":"2024-09-21T02:03:59.835266+0000","last_clean":"2024-09-21T02:03:59.835266+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:03:59.835266+0000","last_undegraded":"2024-09-21T02:03:59.835266+0000","last_fullsized":"2024-09-21T02:03:59.835266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072781+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:03:59.072781+0000","last_peered":"2024-09-21T02:03:59.072781+0000","last_clean":"2024-09-21T02:03:59.072781+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:03:59.072781+0000","last_undegraded":"2024-09-21T02:03:59.072781+0000","last_fullsized":"2024-09-21T02:03:59.072781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075493+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:03:59.075493+0000","last_peered":"2024-09-21T02:03:59.075493+0000","last_clean":"2024-09-21T02:03:59.075493+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:03:59.075493+0000","last_undegraded":"2024-09-21T02:03:59.075493+0000","last_fullsized":"2024-09-21T02:03:59.075493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781505+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:03:59.781505+0000","last_peered":"2024-09-21T02:03:59.781505+0000","last_clean":"2024-09-21T02:03:59.781505+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:03:59.781505+0000","last_undegraded":"2024-09-21T02:03:59.781505+0000","last_fullsized":"2024-09-21T02:03:59.781505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072896+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:03:59.072896+0000","last_peered":"2024-09-21T02:03:59.072896+0000","last_clean":"2024-09-21T02:03:59.072896+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:03:59.072896+0000","last_undegraded":"2024-09-21T02:03:59.072896+0000","last_fullsized":"2024-09-21T02:03:59.072896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076487+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:03:59.076487+0000","last_peered":"2024-09-21T02:03:59.076487+0000","last_clean":"2024-09-21T02:03:59.076487+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:03:59.076487+0000","last_undegraded":"2024-09-21T02:03:59.076487+0000","last_fullsized":"2024-09-21T02:03:59.076487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075934+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:03:59.075934+0000","last_peered":"2024-09-21T02:03:59.075934+0000","last_clean":"2024-09-21T02:03:59.075934+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:03:59.075934+0000","last_undegraded":"2024-09-21T02:03:59.075934+0000","last_fullsized":"2024-09-21T02:03:59.075934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073021+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:03:59.073021+0000","last_peered":"2024-09-21T02:03:59.073021+0000","last_clean":"2024-09-21T02:03:59.073021+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:03:59.073021+0000","last_undegraded":"2024-09-21T02:03:59.073021+0000","last_fullsized":"2024-09-21T02:03:59.073021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073068+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:03:59.073068+0000","last_peered":"2024-09-21T02:03:59.073068+0000","last_clean":"2024-09-21T02:03:59.073068+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:03:59.073068+0000","last_undegraded":"2024-09-21T02:03:59.073068+0000","last_fullsized":"2024-09-21T02:03:59.073068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836949+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:03:59.836949+0000","last_peered":"2024-09-21T02:03:59.836949+0000","last_clean":"2024-09-21T02:03:59.836949+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:03:59.836949+0000","last_undegraded":"2024-09-21T02:03:59.836949+0000","last_fullsized":"2024-09-21T02:03:59.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.067706+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:04:03.067706+0000","last_peered":"2024-09-21T02:04:03.067706+0000","last_clean":"2024-09-21T02:04:03.067706+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:04:03.067706+0000","last_undegraded":"2024-09-21T02:04:03.067706+0000","last_fullsized":"2024-09-21T02:04:03.067706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836206+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:03:59.836206+0000","last_peered":"2024-09-21T02:03:59.836206+0000","last_clean":"2024-09-21T02:03:59.836206+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:03:59.836206+0000","last_undegraded":"2024-09-21T02:03:59.836206+0000","last_fullsized":"2024-09-21T02:03:59.836206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073699+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:03:59.073699+0000","last_peered":"2024-09-21T02:03:59.073699+0000","last_clean":"2024-09-21T02:03:59.073699+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:03:59.073699+0000","last_undegraded":"2024-09-21T02:03:59.073699+0000","last_fullsized":"2024-09-21T02:03:59.073699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781583+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:03:59.781583+0000","last_peered":"2024-09-21T02:03:59.781583+0000","last_clean":"2024-09-21T02:03:59.781583+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:03:59.781583+0000","last_undegraded":"2024-09-21T02:03:59.781583+0000","last_fullsized":"2024-09-21T02:03:59.781583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072755+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:03:59.072755+0000","last_peered":"2024-09-21T02:03:59.072755+0000","last_clean":"2024-09-21T02:03:59.072755+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:03:59.072755+0000","last_undegraded":"2024-09-21T02:03:59.072755+0000","last_fullsized":"2024-09-21T02:03:59.072755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835669+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:03:59.835669+0000","last_peered":"2024-09-21T02:03:59.835669+0000","last_clean":"2024-09-21T02:03:59.835669+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:03:59.835669+0000","last_undegraded":"2024-09-21T02:03:59.835669+0000","last_fullsized":"2024-09-21T02:03:59.835669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072863+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:03:59.072863+0000","last_peered":"2024-09-21T02:03:59.072863+0000","last_clean":"2024-09-21T02:03:59.072863+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:03:59.072863+0000","last_undegraded":"2024-09-21T02:03:59.072863+0000","last_fullsized":"2024-09-21T02:03:59.072863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836615+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:03:59.836615+0000","last_peered":"2024-09-21T02:03:59.836615+0000","last_clean":"2024-09-21T02:03:59.836615+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:03:59.836615+0000","last_undegraded":"2024-09-21T02:03:59.836615+0000","last_fullsized":"2024-09-21T02:03:59.836615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.068565+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:04:03.068565+0000","last_peered":"2024-09-21T02:04:03.068565+0000","last_clean":"2024-09-21T02:04:03.068565+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:04:03.068565+0000","last_undegraded":"2024-09-21T02:04:03.068565+0000","last_fullsized":"2024-09-21T02:04:03.068565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835473+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:03:59.835473+0000","last_peered":"2024-09-21T02:03:59.835473+0000","last_clean":"2024-09-21T02:03:59.835473+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:03:59.835473+0000","last_undegraded":"2024-09-21T02:03:59.835473+0000","last_fullsized":"2024-09-21T02:03:59.835473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076421+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:03:59.076421+0000","last_peered":"2024-09-21T02:03:59.076421+0000","last_clean":"2024-09-21T02:03:59.076421+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:03:59.076421+0000","last_undegraded":"2024-09-21T02:03:59.076421+0000","last_fullsized":"2024-09-21T02:03:59.076421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069209+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:03.069209+0000","last_peered":"2024-09-21T02:04:03.069209+0000","last_clean":"2024-09-21T02:04:03.069209+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:03.069209+0000","last_undegraded":"2024-09-21T02:04:03.069209+0000","last_fullsized":"2024-09-21T02:04:03.069209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075820+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:03:59.075820+0000","last_peered":"2024-09-21T02:03:59.075820+0000","last_clean":"2024-09-21T02:03:59.075820+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:03:59.075820+0000","last_undegraded":"2024-09-21T02:03:59.075820+0000","last_fullsized":"2024-09-21T02:03:59.075820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070016+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:04:03.070016+0000","last_peered":"2024-09-21T02:04:03.070016+0000","last_clean":"2024-09-21T02:04:03.070016+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:04:03.070016+0000","last_undegraded":"2024-09-21T02:04:03.070016+0000","last_fullsized":"2024-09-21T02:04:03.070016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836754+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:03:59.836754+0000","last_peered":"2024-09-21T02:03:59.836754+0000","last_clean":"2024-09-21T02:03:59.836754+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:03:59.836754+0000","last_undegraded":"2024-09-21T02:03:59.836754+0000","last_fullsized":"2024-09-21T02:03:59.836754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837080+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:03:59.837080+0000","last_peered":"2024-09-21T02:03:59.837080+0000","last_clean":"2024-09-21T02:03:59.837080+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:03:59.837080+0000","last_undegraded":"2024-09-21T02:03:59.837080+0000","last_fullsized":"2024-09-21T02:03:59.837080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837178+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:03:59.837178+0000","last_peered":"2024-09-21T02:03:59.837178+0000","last_clean":"2024-09-21T02:03:59.837178+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:03:59.837178+0000","last_undegraded":"2024-09-21T02:03:59.837178+0000","last_fullsized":"2024-09-21T02:03:59.837178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070271+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:04:03.070271+0000","last_peered":"2024-09-21T02:04:03.070271+0000","last_clean":"2024-09-21T02:04:03.070271+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:04:03.070271+0000","last_undegraded":"2024-09-21T02:04:03.070271+0000","last_fullsized":"2024-09-21T02:04:03.070271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":806912,"data_stored":663697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":72837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":72873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":74359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":221184,"data_stored":74359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":663753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:07.913 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:08.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.061+0000 7f16b1fad640 1 -- 172.21.15.88:0/2234028231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0cad60 msgr2=0x7f16ac06cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.061+0000 7f16b1fad640 1 --2- 172.21.15.88:0/2234028231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0cad60 0x7f16ac06cd60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16ac06d9f0 tx=0x7f16a002a5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.062+0000 7f16b1fad640 1 -- 172.21.15.88:0/2234028231 shutdown_connections 2024-09-21T02:04:08.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.062+0000 7f16b1fad640 1 --2- 172.21.15.88:0/2234028231 >> v2:172.21.15.195:3300/0 conn(0x7f16ac0cb170 0x7f16ac06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.062+0000 7f16b1fad640 1 --2- 172.21.15.88:0/2234028231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0cad60 0x7f16ac06cd60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.062+0000 7f16b1fad640 1 --2- 172.21.15.88:0/2234028231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16ac0ccd00 0x7f16ac15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.062+0000 7f16b1fad640 1 -- 172.21.15.88:0/2234028231 >> 172.21.15.88:0/2234028231 conn(0x7f16ac0cd9f0 msgr2=0x7f16ac0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.062+0000 7f16b1fad640 1 -- 172.21.15.88:0/2234028231 shutdown_connections 2024-09-21T02:04:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.062+0000 7f16b1fad640 1 -- 172.21.15.88:0/2234028231 wait complete. 2024-09-21T02:04:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.063+0000 7f16b1fad640 1 Processor -- start 2024-09-21T02:04:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.063+0000 7f16b1fad640 1 -- start start 2024-09-21T02:04:08.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16ac0cad60 0x7f16ac0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16ac0cb170 0x7f16ac158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0ccd00 0x7f16ac15e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac15c640 con 0x7f16ac0ccd00 2024-09-21T02:04:08.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac15c360 con 0x7f16ac0cb170 2024-09-21T02:04:08.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f16ac06dcd0 con 0x7f16ac0cad60 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0ccd00 0x7f16ac15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0ccd00 0x7f16ac15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48424/0 (socket says 172.21.15.88:48424) 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 -- 172.21.15.88:0/4221730750 learned_addr learned my addr 172.21.15.88:0/4221730750 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16aaffd640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16ac0cb170 0x7f16ac158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16ab7fe640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16ac0cad60 0x7f16ac0c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 -- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16ac0cad60 msgr2=0x7f16ac0c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16ac0cad60 0x7f16ac0c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 -- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16ac0cb170 msgr2=0x7f16ac158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16ac0cb170 0x7f16ac158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 -- 172.21.15.88:0/4221730750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac1595f0 con 0x7f16ac0ccd00 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16abfff640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0ccd00 0x7f16ac15e200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f16a400c970 tx=0x7f16a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f1693fff640 1 -- 172.21.15.88:0/4221730750 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4007bb0 con 0x7f16ac0ccd00 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f1693fff640 1 -- 172.21.15.88:0/4221730750 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16a400f450 con 0x7f16ac0ccd00 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac157ca0 con 0x7f16ac0ccd00 2024-09-21T02:04:08.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f1693fff640 1 -- 172.21.15.88:0/4221730750 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4007e60 con 0x7f16ac0ccd00 2024-09-21T02:04:08.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.065+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac15ef00 con 0x7f16ac0ccd00 2024-09-21T02:04:08.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.067+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac0b5c50 con 0x7f16ac0ccd00 2024-09-21T02:04:08.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.067+0000 7f1693fff640 1 -- 172.21.15.88:0/4221730750 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f16a400f8a0 con 0x7f16ac0ccd00 2024-09-21T02:04:08.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.068+0000 7f1693fff640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1698078b90 0x7f169807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.068+0000 7f1693fff640 1 -- 172.21.15.88:0/4221730750 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f16a409b3b0 con 0x7f16ac0ccd00 2024-09-21T02:04:08.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.068+0000 7f16ab7fe640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1698078b90 0x7f169807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.068+0000 7f16ab7fe640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1698078b90 0x7f169807b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f169c005c40 tx=0x7f169c00a530 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:08.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.072+0000 7f1693fff640 1 -- 172.21.15.88:0/4221730750 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f16ac0b5c50 con 0x7f16ac0ccd00 2024-09-21T02:04:08.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.253+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16ac15f970 con 0x7f1698078b90 2024-09-21T02:04:08.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.257+0000 7f1693fff640 1 -- 172.21.15.88:0/4221730750 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251824 (secure 0 0 0) 0x7f16ac15f970 con 0x7f1698078b90 2024-09-21T02:04:08.261 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:08.262 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:04:08.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1698078b90 msgr2=0x7f169807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1698078b90 0x7f169807b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f169c005c40 tx=0x7f169c00a530 comp rx=0 tx=0).stop 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0ccd00 msgr2=0x7f16ac15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0ccd00 0x7f16ac15e200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f16a400c970 tx=0x7f16a400ce30 comp rx=0 tx=0).stop 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 shutdown_connections 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1698078b90 0x7f169807b0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16ac0ccd00 0x7f16ac15e200 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16ac0cb170 0x7f16ac158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 --2- 172.21.15.88:0/4221730750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16ac0cad60 0x7f16ac0c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.260+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 >> 172.21.15.88:0/4221730750 conn(0x7f16ac0cd9f0 msgr2=0x7f16ac1e6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.261+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 shutdown_connections 2024-09-21T02:04:08.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.261+0000 7f16b1fad640 1 -- 172.21.15.88:0/4221730750 wait complete. 2024-09-21T02:04:08.273 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-21T02:04:06.820090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239848,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734872,"statfs":{"total":773094113280,"available":772848508928,"internally_reserved":0,"allocated":2924544,"data_stored":1767532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.759880"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074727+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:03:59.074727+0000","last_peered":"2024-09-21T02:03:59.074727+0000","last_clean":"2024-09-21T02:03:59.074727+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:03:59.074727+0000","last_undegraded":"2024-09-21T02:03:59.074727+0000","last_fullsized":"2024-09-21T02:03:59.074727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073213+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:03:59.073213+0000","last_peered":"2024-09-21T02:03:59.073213+0000","last_clean":"2024-09-21T02:03:59.073213+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:03:59.073213+0000","last_undegraded":"2024-09-21T02:03:59.073213+0000","last_fullsized":"2024-09-21T02:03:59.073213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836539+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:03:59.836539+0000","last_peered":"2024-09-21T02:03:59.836539+0000","last_clean":"2024-09-21T02:03:59.836539+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:03:59.836539+0000","last_undegraded":"2024-09-21T02:03:59.836539+0000","last_fullsized":"2024-09-21T02:03:59.836539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075220+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:03:59.075220+0000","last_peered":"2024-09-21T02:03:59.075220+0000","last_clean":"2024-09-21T02:03:59.075220+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:03:59.075220+0000","last_undegraded":"2024-09-21T02:03:59.075220+0000","last_fullsized":"2024-09-21T02:03:59.075220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074990+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:03:59.074990+0000","last_peered":"2024-09-21T02:03:59.074990+0000","last_clean":"2024-09-21T02:03:59.074990+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:03:59.074990+0000","last_undegraded":"2024-09-21T02:03:59.074990+0000","last_fullsized":"2024-09-21T02:03:59.074990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074223+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:03:59.074223+0000","last_peered":"2024-09-21T02:03:59.074223+0000","last_clean":"2024-09-21T02:03:59.074223+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:03:59.074223+0000","last_undegraded":"2024-09-21T02:03:59.074223+0000","last_fullsized":"2024-09-21T02:03:59.074223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074095+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:03:59.074095+0000","last_peered":"2024-09-21T02:03:59.074095+0000","last_clean":"2024-09-21T02:03:59.074095+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:03:59.074095+0000","last_undegraded":"2024-09-21T02:03:59.074095+0000","last_fullsized":"2024-09-21T02:03:59.074095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835347+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:03:59.835347+0000","last_peered":"2024-09-21T02:03:59.835347+0000","last_clean":"2024-09-21T02:03:59.835347+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:03:59.835347+0000","last_undegraded":"2024-09-21T02:03:59.835347+0000","last_fullsized":"2024-09-21T02:03:59.835347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074637+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:03:59.074637+0000","last_peered":"2024-09-21T02:03:59.074637+0000","last_clean":"2024-09-21T02:03:59.074637+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:03:59.074637+0000","last_undegraded":"2024-09-21T02:03:59.074637+0000","last_fullsized":"2024-09-21T02:03:59.074637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835966+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:03:59.835966+0000","last_peered":"2024-09-21T02:03:59.835966+0000","last_clean":"2024-09-21T02:03:59.835966+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:03:59.835966+0000","last_undegraded":"2024-09-21T02:03:59.835966+0000","last_fullsized":"2024-09-21T02:03:59.835966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835587+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:03:59.835587+0000","last_peered":"2024-09-21T02:03:59.835587+0000","last_clean":"2024-09-21T02:03:59.835587+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:03:59.835587+0000","last_undegraded":"2024-09-21T02:03:59.835587+0000","last_fullsized":"2024-09-21T02:03:59.835587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781637+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:03:59.781637+0000","last_peered":"2024-09-21T02:03:59.781637+0000","last_clean":"2024-09-21T02:03:59.781637+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:03:59.781637+0000","last_undegraded":"2024-09-21T02:03:59.781637+0000","last_fullsized":"2024-09-21T02:03:59.781637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835918+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:03:59.835918+0000","last_peered":"2024-09-21T02:03:59.835918+0000","last_clean":"2024-09-21T02:03:59.835918+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:03:59.835918+0000","last_undegraded":"2024-09-21T02:03:59.835918+0000","last_fullsized":"2024-09-21T02:03:59.835918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836186+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:03:59.836186+0000","last_peered":"2024-09-21T02:03:59.836186+0000","last_clean":"2024-09-21T02:03:59.836186+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:03:59.836186+0000","last_undegraded":"2024-09-21T02:03:59.836186+0000","last_fullsized":"2024-09-21T02:03:59.836186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072763+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:03:59.072763+0000","last_peered":"2024-09-21T02:03:59.072763+0000","last_clean":"2024-09-21T02:03:59.072763+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:03:59.072763+0000","last_undegraded":"2024-09-21T02:03:59.072763+0000","last_fullsized":"2024-09-21T02:03:59.072763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075548+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:03:59.075548+0000","last_peered":"2024-09-21T02:03:59.075548+0000","last_clean":"2024-09-21T02:03:59.075548+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:03:59.075548+0000","last_undegraded":"2024-09-21T02:03:59.075548+0000","last_fullsized":"2024-09-21T02:03:59.075548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075910+0000","last_change":"2024-09-21T02:03:58.074570+0000","last_active":"2024-09-21T02:03:59.075910+0000","last_peered":"2024-09-21T02:03:59.075910+0000","last_clean":"2024-09-21T02:03:59.075910+0000","last_became_active":"2024-09-21T02:03:58.074271+0000","last_became_peered":"2024-09-21T02:03:58.074271+0000","last_unstale":"2024-09-21T02:03:59.075910+0000","last_undegraded":"2024-09-21T02:03:59.075910+0000","last_fullsized":"2024-09-21T02:03:59.075910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.292792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835423+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:03:59.835423+0000","last_peered":"2024-09-21T02:03:59.835423+0000","last_clean":"2024-09-21T02:03:59.835423+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:03:59.835423+0000","last_undegraded":"2024-09-21T02:03:59.835423+0000","last_fullsized":"2024-09-21T02:03:59.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076132+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:03:59.076132+0000","last_peered":"2024-09-21T02:03:59.076132+0000","last_clean":"2024-09-21T02:03:59.076132+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:03:59.076132+0000","last_undegraded":"2024-09-21T02:03:59.076132+0000","last_fullsized":"2024-09-21T02:03:59.076132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835510+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:03:59.835510+0000","last_peered":"2024-09-21T02:03:59.835510+0000","last_clean":"2024-09-21T02:03:59.835510+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:03:59.835510+0000","last_undegraded":"2024-09-21T02:03:59.835510+0000","last_fullsized":"2024-09-21T02:03:59.835510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836526+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:03:59.836526+0000","last_peered":"2024-09-21T02:03:59.836526+0000","last_clean":"2024-09-21T02:03:59.836526+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:03:59.836526+0000","last_undegraded":"2024-09-21T02:03:59.836526+0000","last_fullsized":"2024-09-21T02:03:59.836526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075251+0000","last_change":"2024-09-21T02:03:58.076378+0000","last_active":"2024-09-21T02:03:59.075251+0000","last_peered":"2024-09-21T02:03:59.075251+0000","last_clean":"2024-09-21T02:03:59.075251+0000","last_became_active":"2024-09-21T02:03:58.076075+0000","last_became_peered":"2024-09-21T02:03:58.076075+0000","last_unstale":"2024-09-21T02:03:59.075251+0000","last_undegraded":"2024-09-21T02:03:59.075251+0000","last_fullsized":"2024-09-21T02:03:59.075251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:54.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835596+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:03:59.835596+0000","last_peered":"2024-09-21T02:03:59.835596+0000","last_clean":"2024-09-21T02:03:59.835596+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:03:59.835596+0000","last_undegraded":"2024-09-21T02:03:59.835596+0000","last_fullsized":"2024-09-21T02:03:59.835596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074571+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:03:59.074571+0000","last_peered":"2024-09-21T02:03:59.074571+0000","last_clean":"2024-09-21T02:03:59.074571+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:03:59.074571+0000","last_undegraded":"2024-09-21T02:03:59.074571+0000","last_fullsized":"2024-09-21T02:03:59.074571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836762+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:03:59.836762+0000","last_peered":"2024-09-21T02:03:59.836762+0000","last_clean":"2024-09-21T02:03:59.836762+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:03:59.836762+0000","last_undegraded":"2024-09-21T02:03:59.836762+0000","last_fullsized":"2024-09-21T02:03:59.836762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835955+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:03:59.835955+0000","last_peered":"2024-09-21T02:03:59.835955+0000","last_clean":"2024-09-21T02:03:59.835955+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:03:59.835955+0000","last_undegraded":"2024-09-21T02:03:59.835955+0000","last_fullsized":"2024-09-21T02:03:59.835955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073053+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:03:59.073053+0000","last_peered":"2024-09-21T02:03:59.073053+0000","last_clean":"2024-09-21T02:03:59.073053+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:03:59.073053+0000","last_undegraded":"2024-09-21T02:03:59.073053+0000","last_fullsized":"2024-09-21T02:03:59.073053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836383+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:03:59.836383+0000","last_peered":"2024-09-21T02:03:59.836383+0000","last_clean":"2024-09-21T02:03:59.836383+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:03:59.836383+0000","last_undegraded":"2024-09-21T02:03:59.836383+0000","last_fullsized":"2024-09-21T02:03:59.836383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836861+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:03:59.836861+0000","last_peered":"2024-09-21T02:03:59.836861+0000","last_clean":"2024-09-21T02:03:59.836861+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:03:59.836861+0000","last_undegraded":"2024-09-21T02:03:59.836861+0000","last_fullsized":"2024-09-21T02:03:59.836861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075554+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:03:59.075554+0000","last_peered":"2024-09-21T02:03:59.075554+0000","last_clean":"2024-09-21T02:03:59.075554+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:03:59.075554+0000","last_undegraded":"2024-09-21T02:03:59.075554+0000","last_fullsized":"2024-09-21T02:03:59.075554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835391+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:03:59.835391+0000","last_peered":"2024-09-21T02:03:59.835391+0000","last_clean":"2024-09-21T02:03:59.835391+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:03:59.835391+0000","last_undegraded":"2024-09-21T02:03:59.835391+0000","last_fullsized":"2024-09-21T02:03:59.835391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835439+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:03:59.835439+0000","last_peered":"2024-09-21T02:03:59.835439+0000","last_clean":"2024-09-21T02:03:59.835439+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:03:59.835439+0000","last_undegraded":"2024-09-21T02:03:59.835439+0000","last_fullsized":"2024-09-21T02:03:59.835439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781525+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:03:59.781525+0000","last_peered":"2024-09-21T02:03:59.781525+0000","last_clean":"2024-09-21T02:03:59.781525+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:03:59.781525+0000","last_undegraded":"2024-09-21T02:03:59.781525+0000","last_fullsized":"2024-09-21T02:03:59.781525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836732+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:03:59.836732+0000","last_peered":"2024-09-21T02:03:59.836732+0000","last_clean":"2024-09-21T02:03:59.836732+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:03:59.836732+0000","last_undegraded":"2024-09-21T02:03:59.836732+0000","last_fullsized":"2024-09-21T02:03:59.836732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836856+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:03:59.836856+0000","last_peered":"2024-09-21T02:03:59.836856+0000","last_clean":"2024-09-21T02:03:59.836856+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:03:59.836856+0000","last_undegraded":"2024-09-21T02:03:59.836856+0000","last_fullsized":"2024-09-21T02:03:59.836856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835620+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:03:59.835620+0000","last_peered":"2024-09-21T02:03:59.835620+0000","last_clean":"2024-09-21T02:03:59.835620+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:03:59.835620+0000","last_undegraded":"2024-09-21T02:03:59.835620+0000","last_fullsized":"2024-09-21T02:03:59.835620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835432+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:03:59.835432+0000","last_peered":"2024-09-21T02:03:59.835432+0000","last_clean":"2024-09-21T02:03:59.835432+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:03:59.835432+0000","last_undegraded":"2024-09-21T02:03:59.835432+0000","last_fullsized":"2024-09-21T02:03:59.835432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835573+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:03:59.835573+0000","last_peered":"2024-09-21T02:03:59.835573+0000","last_clean":"2024-09-21T02:03:59.835573+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:03:59.835573+0000","last_undegraded":"2024-09-21T02:03:59.835573+0000","last_fullsized":"2024-09-21T02:03:59.835573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074657+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:03:59.074657+0000","last_peered":"2024-09-21T02:03:59.074657+0000","last_clean":"2024-09-21T02:03:59.074657+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:03:59.074657+0000","last_undegraded":"2024-09-21T02:03:59.074657+0000","last_fullsized":"2024-09-21T02:03:59.074657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072701+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:03:59.072701+0000","last_peered":"2024-09-21T02:03:59.072701+0000","last_clean":"2024-09-21T02:03:59.072701+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:03:59.072701+0000","last_undegraded":"2024-09-21T02:03:59.072701+0000","last_fullsized":"2024-09-21T02:03:59.072701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835536+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:03:59.835536+0000","last_peered":"2024-09-21T02:03:59.835536+0000","last_clean":"2024-09-21T02:03:59.835536+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:03:59.835536+0000","last_undegraded":"2024-09-21T02:03:59.835536+0000","last_fullsized":"2024-09-21T02:03:59.835536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074748+0000","last_change":"2024-09-21T02:03:58.075434+0000","last_active":"2024-09-21T02:03:59.074748+0000","last_peered":"2024-09-21T02:03:59.074748+0000","last_clean":"2024-09-21T02:03:59.074748+0000","last_became_active":"2024-09-21T02:03:58.075130+0000","last_became_peered":"2024-09-21T02:03:58.075130+0000","last_unstale":"2024-09-21T02:03:59.074748+0000","last_undegraded":"2024-09-21T02:03:59.074748+0000","last_fullsized":"2024-09-21T02:03:59.074748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:56.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074818+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:03:59.074818+0000","last_peered":"2024-09-21T02:03:59.074818+0000","last_clean":"2024-09-21T02:03:59.074818+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:03:59.074818+0000","last_undegraded":"2024-09-21T02:03:59.074818+0000","last_fullsized":"2024-09-21T02:03:59.074818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836482+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:03:59.836482+0000","last_peered":"2024-09-21T02:03:59.836482+0000","last_clean":"2024-09-21T02:03:59.836482+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:03:59.836482+0000","last_undegraded":"2024-09-21T02:03:59.836482+0000","last_fullsized":"2024-09-21T02:03:59.836482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069302+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:04:03.069302+0000","last_peered":"2024-09-21T02:04:03.069302+0000","last_clean":"2024-09-21T02:04:03.069302+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:04:03.069302+0000","last_undegraded":"2024-09-21T02:04:03.069302+0000","last_fullsized":"2024-09-21T02:04:03.069302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781695+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:03:59.781695+0000","last_peered":"2024-09-21T02:03:59.781695+0000","last_clean":"2024-09-21T02:03:59.781695+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:03:59.781695+0000","last_undegraded":"2024-09-21T02:03:59.781695+0000","last_fullsized":"2024-09-21T02:03:59.781695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.071755+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:04:03.071755+0000","last_peered":"2024-09-21T02:04:03.071755+0000","last_clean":"2024-09-21T02:04:03.071755+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:04:03.071755+0000","last_undegraded":"2024-09-21T02:04:03.071755+0000","last_fullsized":"2024-09-21T02:04:03.071755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075568+0000","last_change":"2024-09-21T02:03:58.080947+0000","last_active":"2024-09-21T02:03:59.075568+0000","last_peered":"2024-09-21T02:03:59.075568+0000","last_clean":"2024-09-21T02:03:59.075568+0000","last_became_active":"2024-09-21T02:03:58.080632+0000","last_became_peered":"2024-09-21T02:03:58.080632+0000","last_unstale":"2024-09-21T02:03:59.075568+0000","last_undegraded":"2024-09-21T02:03:59.075568+0000","last_fullsized":"2024-09-21T02:03:59.075568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:40.307689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070474+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:03.070474+0000","last_peered":"2024-09-21T02:04:03.070474+0000","last_clean":"2024-09-21T02:04:03.070474+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:03.070474+0000","last_undegraded":"2024-09-21T02:04:03.070474+0000","last_fullsized":"2024-09-21T02:04:03.070474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836343+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:03:59.836343+0000","last_peered":"2024-09-21T02:03:59.836343+0000","last_clean":"2024-09-21T02:03:59.836343+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:03:59.836343+0000","last_undegraded":"2024-09-21T02:03:59.836343+0000","last_fullsized":"2024-09-21T02:03:59.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075336+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:03:59.075336+0000","last_peered":"2024-09-21T02:03:59.075336+0000","last_clean":"2024-09-21T02:03:59.075336+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:03:59.075336+0000","last_undegraded":"2024-09-21T02:03:59.075336+0000","last_fullsized":"2024-09-21T02:03:59.075336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075210+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:03:59.075210+0000","last_peered":"2024-09-21T02:03:59.075210+0000","last_clean":"2024-09-21T02:03:59.075210+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:03:59.075210+0000","last_undegraded":"2024-09-21T02:03:59.075210+0000","last_fullsized":"2024-09-21T02:03:59.075210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835374+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:03:59.835374+0000","last_peered":"2024-09-21T02:03:59.835374+0000","last_clean":"2024-09-21T02:03:59.835374+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:03:59.835374+0000","last_undegraded":"2024-09-21T02:03:59.835374+0000","last_fullsized":"2024-09-21T02:03:59.835374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835641+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:03:59.835641+0000","last_peered":"2024-09-21T02:03:59.835641+0000","last_clean":"2024-09-21T02:03:59.835641+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:03:59.835641+0000","last_undegraded":"2024-09-21T02:03:59.835641+0000","last_fullsized":"2024-09-21T02:03:59.835641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.072054+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:04:03.072054+0000","last_peered":"2024-09-21T02:04:03.072054+0000","last_clean":"2024-09-21T02:04:03.072054+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:04:03.072054+0000","last_undegraded":"2024-09-21T02:04:03.072054+0000","last_fullsized":"2024-09-21T02:04:03.072054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074582+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:03:59.074582+0000","last_peered":"2024-09-21T02:03:59.074582+0000","last_clean":"2024-09-21T02:03:59.074582+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:03:59.074582+0000","last_undegraded":"2024-09-21T02:03:59.074582+0000","last_fullsized":"2024-09-21T02:03:59.074582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781594+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:03:59.781594+0000","last_peered":"2024-09-21T02:03:59.781594+0000","last_clean":"2024-09-21T02:03:59.781594+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:03:59.781594+0000","last_undegraded":"2024-09-21T02:03:59.781594+0000","last_fullsized":"2024-09-21T02:03:59.781594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074555+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:03:59.074555+0000","last_peered":"2024-09-21T02:03:59.074555+0000","last_clean":"2024-09-21T02:03:59.074555+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:03:59.074555+0000","last_undegraded":"2024-09-21T02:03:59.074555+0000","last_fullsized":"2024-09-21T02:03:59.074555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.071532+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:03.071532+0000","last_peered":"2024-09-21T02:04:03.071532+0000","last_clean":"2024-09-21T02:04:03.071532+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:03.071532+0000","last_undegraded":"2024-09-21T02:04:03.071532+0000","last_fullsized":"2024-09-21T02:04:03.071532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074911+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:03:59.074911+0000","last_peered":"2024-09-21T02:03:59.074911+0000","last_clean":"2024-09-21T02:03:59.074911+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:03:59.074911+0000","last_undegraded":"2024-09-21T02:03:59.074911+0000","last_fullsized":"2024-09-21T02:03:59.074911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073109+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:03:59.073109+0000","last_peered":"2024-09-21T02:03:59.073109+0000","last_clean":"2024-09-21T02:03:59.073109+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:03:59.073109+0000","last_undegraded":"2024-09-21T02:03:59.073109+0000","last_fullsized":"2024-09-21T02:03:59.073109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075681+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:03:59.075681+0000","last_peered":"2024-09-21T02:03:59.075681+0000","last_clean":"2024-09-21T02:03:59.075681+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:03:59.075681+0000","last_undegraded":"2024-09-21T02:03:59.075681+0000","last_fullsized":"2024-09-21T02:03:59.075681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835624+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:03:59.835624+0000","last_peered":"2024-09-21T02:03:59.835624+0000","last_clean":"2024-09-21T02:03:59.835624+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:03:59.835624+0000","last_undegraded":"2024-09-21T02:03:59.835624+0000","last_fullsized":"2024-09-21T02:03:59.835624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075817+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:03:59.075817+0000","last_peered":"2024-09-21T02:03:59.075817+0000","last_clean":"2024-09-21T02:03:59.075817+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:03:59.075817+0000","last_undegraded":"2024-09-21T02:03:59.075817+0000","last_fullsized":"2024-09-21T02:03:59.075817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074041+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:03:59.074041+0000","last_peered":"2024-09-21T02:03:59.074041+0000","last_clean":"2024-09-21T02:03:59.074041+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:03:59.074041+0000","last_undegraded":"2024-09-21T02:03:59.074041+0000","last_fullsized":"2024-09-21T02:03:59.074041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074376+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:03:59.074376+0000","last_peered":"2024-09-21T02:03:59.074376+0000","last_clean":"2024-09-21T02:03:59.074376+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:03:59.074376+0000","last_undegraded":"2024-09-21T02:03:59.074376+0000","last_fullsized":"2024-09-21T02:03:59.074376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835923+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:03:59.835923+0000","last_peered":"2024-09-21T02:03:59.835923+0000","last_clean":"2024-09-21T02:03:59.835923+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:03:59.835923+0000","last_undegraded":"2024-09-21T02:03:59.835923+0000","last_fullsized":"2024-09-21T02:03:59.835923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073892+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:03:59.073892+0000","last_peered":"2024-09-21T02:03:59.073892+0000","last_clean":"2024-09-21T02:03:59.073892+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:03:59.073892+0000","last_undegraded":"2024-09-21T02:03:59.073892+0000","last_fullsized":"2024-09-21T02:03:59.073892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835986+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:03:59.835986+0000","last_peered":"2024-09-21T02:03:59.835986+0000","last_clean":"2024-09-21T02:03:59.835986+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:03:59.835986+0000","last_undegraded":"2024-09-21T02:03:59.835986+0000","last_fullsized":"2024-09-21T02:03:59.835986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:05.466556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835266+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:03:59.835266+0000","last_peered":"2024-09-21T02:03:59.835266+0000","last_clean":"2024-09-21T02:03:59.835266+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:03:59.835266+0000","last_undegraded":"2024-09-21T02:03:59.835266+0000","last_fullsized":"2024-09-21T02:03:59.835266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072781+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:03:59.072781+0000","last_peered":"2024-09-21T02:03:59.072781+0000","last_clean":"2024-09-21T02:03:59.072781+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:03:59.072781+0000","last_undegraded":"2024-09-21T02:03:59.072781+0000","last_fullsized":"2024-09-21T02:03:59.072781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075493+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:03:59.075493+0000","last_peered":"2024-09-21T02:03:59.075493+0000","last_clean":"2024-09-21T02:03:59.075493+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:03:59.075493+0000","last_undegraded":"2024-09-21T02:03:59.075493+0000","last_fullsized":"2024-09-21T02:03:59.075493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781505+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:03:59.781505+0000","last_peered":"2024-09-21T02:03:59.781505+0000","last_clean":"2024-09-21T02:03:59.781505+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:03:59.781505+0000","last_undegraded":"2024-09-21T02:03:59.781505+0000","last_fullsized":"2024-09-21T02:03:59.781505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072896+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:03:59.072896+0000","last_peered":"2024-09-21T02:03:59.072896+0000","last_clean":"2024-09-21T02:03:59.072896+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:03:59.072896+0000","last_undegraded":"2024-09-21T02:03:59.072896+0000","last_fullsized":"2024-09-21T02:03:59.072896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076487+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:03:59.076487+0000","last_peered":"2024-09-21T02:03:59.076487+0000","last_clean":"2024-09-21T02:03:59.076487+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:03:59.076487+0000","last_undegraded":"2024-09-21T02:03:59.076487+0000","last_fullsized":"2024-09-21T02:03:59.076487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075934+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:03:59.075934+0000","last_peered":"2024-09-21T02:03:59.075934+0000","last_clean":"2024-09-21T02:03:59.075934+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:03:59.075934+0000","last_undegraded":"2024-09-21T02:03:59.075934+0000","last_fullsized":"2024-09-21T02:03:59.075934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073021+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:03:59.073021+0000","last_peered":"2024-09-21T02:03:59.073021+0000","last_clean":"2024-09-21T02:03:59.073021+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:03:59.073021+0000","last_undegraded":"2024-09-21T02:03:59.073021+0000","last_fullsized":"2024-09-21T02:03:59.073021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073068+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:03:59.073068+0000","last_peered":"2024-09-21T02:03:59.073068+0000","last_clean":"2024-09-21T02:03:59.073068+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:03:59.073068+0000","last_undegraded":"2024-09-21T02:03:59.073068+0000","last_fullsized":"2024-09-21T02:03:59.073068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836949+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:03:59.836949+0000","last_peered":"2024-09-21T02:03:59.836949+0000","last_clean":"2024-09-21T02:03:59.836949+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:03:59.836949+0000","last_undegraded":"2024-09-21T02:03:59.836949+0000","last_fullsized":"2024-09-21T02:03:59.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.067706+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:04:03.067706+0000","last_peered":"2024-09-21T02:04:03.067706+0000","last_clean":"2024-09-21T02:04:03.067706+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:04:03.067706+0000","last_undegraded":"2024-09-21T02:04:03.067706+0000","last_fullsized":"2024-09-21T02:04:03.067706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836206+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:03:59.836206+0000","last_peered":"2024-09-21T02:03:59.836206+0000","last_clean":"2024-09-21T02:03:59.836206+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:03:59.836206+0000","last_undegraded":"2024-09-21T02:03:59.836206+0000","last_fullsized":"2024-09-21T02:03:59.836206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073699+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:03:59.073699+0000","last_peered":"2024-09-21T02:03:59.073699+0000","last_clean":"2024-09-21T02:03:59.073699+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:03:59.073699+0000","last_undegraded":"2024-09-21T02:03:59.073699+0000","last_fullsized":"2024-09-21T02:03:59.073699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781583+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:03:59.781583+0000","last_peered":"2024-09-21T02:03:59.781583+0000","last_clean":"2024-09-21T02:03:59.781583+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:03:59.781583+0000","last_undegraded":"2024-09-21T02:03:59.781583+0000","last_fullsized":"2024-09-21T02:03:59.781583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072755+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:03:59.072755+0000","last_peered":"2024-09-21T02:03:59.072755+0000","last_clean":"2024-09-21T02:03:59.072755+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:03:59.072755+0000","last_undegraded":"2024-09-21T02:03:59.072755+0000","last_fullsized":"2024-09-21T02:03:59.072755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835669+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:03:59.835669+0000","last_peered":"2024-09-21T02:03:59.835669+0000","last_clean":"2024-09-21T02:03:59.835669+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:03:59.835669+0000","last_undegraded":"2024-09-21T02:03:59.835669+0000","last_fullsized":"2024-09-21T02:03:59.835669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072863+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:03:59.072863+0000","last_peered":"2024-09-21T02:03:59.072863+0000","last_clean":"2024-09-21T02:03:59.072863+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:03:59.072863+0000","last_undegraded":"2024-09-21T02:03:59.072863+0000","last_fullsized":"2024-09-21T02:03:59.072863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836615+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:03:59.836615+0000","last_peered":"2024-09-21T02:03:59.836615+0000","last_clean":"2024-09-21T02:03:59.836615+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:03:59.836615+0000","last_undegraded":"2024-09-21T02:03:59.836615+0000","last_fullsized":"2024-09-21T02:03:59.836615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.068565+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:04:03.068565+0000","last_peered":"2024-09-21T02:04:03.068565+0000","last_clean":"2024-09-21T02:04:03.068565+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:04:03.068565+0000","last_undegraded":"2024-09-21T02:04:03.068565+0000","last_fullsized":"2024-09-21T02:04:03.068565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835473+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:03:59.835473+0000","last_peered":"2024-09-21T02:03:59.835473+0000","last_clean":"2024-09-21T02:03:59.835473+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:03:59.835473+0000","last_undegraded":"2024-09-21T02:03:59.835473+0000","last_fullsized":"2024-09-21T02:03:59.835473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076421+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:03:59.076421+0000","last_peered":"2024-09-21T02:03:59.076421+0000","last_clean":"2024-09-21T02:03:59.076421+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:03:59.076421+0000","last_undegraded":"2024-09-21T02:03:59.076421+0000","last_fullsized":"2024-09-21T02:03:59.076421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069209+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:03.069209+0000","last_peered":"2024-09-21T02:04:03.069209+0000","last_clean":"2024-09-21T02:04:03.069209+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:03.069209+0000","last_undegraded":"2024-09-21T02:04:03.069209+0000","last_fullsized":"2024-09-21T02:04:03.069209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075820+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:03:59.075820+0000","last_peered":"2024-09-21T02:03:59.075820+0000","last_clean":"2024-09-21T02:03:59.075820+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:03:59.075820+0000","last_undegraded":"2024-09-21T02:03:59.075820+0000","last_fullsized":"2024-09-21T02:03:59.075820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070016+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:04:03.070016+0000","last_peered":"2024-09-21T02:04:03.070016+0000","last_clean":"2024-09-21T02:04:03.070016+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:04:03.070016+0000","last_undegraded":"2024-09-21T02:04:03.070016+0000","last_fullsized":"2024-09-21T02:04:03.070016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836754+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:03:59.836754+0000","last_peered":"2024-09-21T02:03:59.836754+0000","last_clean":"2024-09-21T02:03:59.836754+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:03:59.836754+0000","last_undegraded":"2024-09-21T02:03:59.836754+0000","last_fullsized":"2024-09-21T02:03:59.836754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837080+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:03:59.837080+0000","last_peered":"2024-09-21T02:03:59.837080+0000","last_clean":"2024-09-21T02:03:59.837080+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:03:59.837080+0000","last_undegraded":"2024-09-21T02:03:59.837080+0000","last_fullsized":"2024-09-21T02:03:59.837080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837178+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:03:59.837178+0000","last_peered":"2024-09-21T02:03:59.837178+0000","last_clean":"2024-09-21T02:03:59.837178+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:03:59.837178+0000","last_undegraded":"2024-09-21T02:03:59.837178+0000","last_fullsized":"2024-09-21T02:03:59.837178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070271+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:04:03.070271+0000","last_peered":"2024-09-21T02:04:03.070271+0000","last_clean":"2024-09-21T02:04:03.070271+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:04:03.070271+0000","last_undegraded":"2024-09-21T02:04:03.070271+0000","last_fullsized":"2024-09-21T02:04:03.070271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":806912,"data_stored":663697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":72837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":72873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":74359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":221184,"data_stored":74359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":663753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:08.277 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:04:08.277 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T02:04:08.277 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T02:04:08.277 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T02:04:08.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.422+0000 7f2442c9b640 1 -- 172.21.15.88:0/3843460244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 msgr2=0x7f243c06cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.422+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3843460244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c06cd60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f243c06d9f0 tx=0x7f242c02a5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.423+0000 7f2442c9b640 1 -- 172.21.15.88:0/3843460244 shutdown_connections 2024-09-21T02:04:08.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.423+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3843460244 >> v2:172.21.15.195:3300/0 conn(0x7f243c0cb170 0x7f243c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.423+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3843460244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c06cd60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.423+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3843460244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f243c0ccd00 0x7f243c15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.423+0000 7f2442c9b640 1 -- 172.21.15.88:0/3843460244 >> 172.21.15.88:0/3843460244 conn(0x7f243c0cd9f0 msgr2=0x7f243c0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:08.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.423+0000 7f2442c9b640 1 -- 172.21.15.88:0/3843460244 shutdown_connections 2024-09-21T02:04:08.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.423+0000 7f2442c9b640 1 -- 172.21.15.88:0/3843460244 wait complete. 2024-09-21T02:04:08.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.424+0000 7f2442c9b640 1 Processor -- start 2024-09-21T02:04:08.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.424+0000 7f2442c9b640 1 -- start start 2024-09-21T02:04:08.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c1f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f243c0cb170 0x7f243c0c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f243c0ccd00 0x7f243c0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f243c15c640 con 0x7f243c0cad60 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f243c15c0e0 con 0x7f243c0cb170 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f243c06dcd0 con 0x7f243c0ccd00 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c1f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c1f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48444/0 (socket says 172.21.15.88:48444) 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 -- 172.21.15.88:0/3650212134 learned_addr learned my addr 172.21.15.88:0/3650212134 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2433fff640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f243c0cb170 0x7f243c0c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2441238640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f243c0ccd00 0x7f243c0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 -- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f243c0ccd00 msgr2=0x7f243c0c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f243c0ccd00 0x7f243c0c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 -- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f243c0cb170 msgr2=0x7f243c0c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f243c0cb170 0x7f243c0c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 -- 172.21.15.88:0/3650212134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c1e6ab0 con 0x7f243c0cad60 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2440a37640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c1f5700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f243800b410 tx=0x7f243800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f24317fa640 1 -- 172.21.15.88:0/3650212134 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438015020 con 0x7f243c0cad60 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f24317fa640 1 -- 172.21.15.88:0/3650212134 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2438011940 con 0x7f243c0cad60 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c1e6d70 con 0x7f243c0cad60 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f24317fa640 1 -- 172.21.15.88:0/3650212134 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438011d90 con 0x7f243c0cad60 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.426+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c1f2500 con 0x7f243c0cad60 2024-09-21T02:04:08.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.428+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c15f970 con 0x7f243c0cad60 2024-09-21T02:04:08.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.428+0000 7f24317fa640 1 -- 172.21.15.88:0/3650212134 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2438018020 con 0x7f243c0cad60 2024-09-21T02:04:08.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.428+0000 7f24317fa640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2428078b90 0x7f242807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:08.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.429+0000 7f24317fa640 1 -- 172.21.15.88:0/3650212134 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f243809ac50 con 0x7f243c0cad60 2024-09-21T02:04:08.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.429+0000 7f2433fff640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2428078b90 0x7f242807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:08.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.429+0000 7f2433fff640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2428078b90 0x7f242807b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f242c033000 tx=0x7f242c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:08.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.433+0000 7f24317fa640 1 -- 172.21.15.88:0/3650212134 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f24380a0020 con 0x7f243c0cad60 2024-09-21T02:04:08.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.644+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f243c155f30 con 0x7f243c0cad60 2024-09-21T02:04:08.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.645+0000 7f24317fa640 1 -- 172.21.15.88:0/3650212134 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f243801f020 con 0x7f243c0cad60 2024-09-21T02:04:08.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:08.646 INFO:teuthology.orchestra.run.smithi088.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2428078b90 msgr2=0x7f242807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2428078b90 0x7f242807b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f242c033000 tx=0x7f242c004730 comp rx=0 tx=0).stop 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 msgr2=0x7f243c1f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c1f5700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f243800b410 tx=0x7f243800b8d0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 shutdown_connections 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2428078b90 0x7f242807b0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f243c0ccd00 0x7f243c0c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f243c0cb170 0x7f243c0c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 --2- 172.21.15.88:0/3650212134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f243c0cad60 0x7f243c1f5700 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:08.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 >> 172.21.15.88:0/3650212134 conn(0x7f243c0cd9f0 msgr2=0x7f243c159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:08.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 shutdown_connections 2024-09-21T02:04:08.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:08.647+0000 7f2442c9b640 1 -- 172.21.15.88:0/3650212134 wait complete. 2024-09-21T02:04:11.660 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T02:04:11.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 -- 172.21.15.88:0/3218298492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 msgr2=0x7f014c156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 --2- 172.21.15.88:0/3218298492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c156df0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f014c0c9b30 tx=0x7f014402a5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 -- 172.21.15.88:0/3218298492 shutdown_connections 2024-09-21T02:04:11.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 --2- 172.21.15.88:0/3218298492 >> v2:172.21.15.195:3300/0 conn(0x7f014c0c8a00 0x7f014c0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 --2- 172.21.15.88:0/3218298492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f014c0c4f10 0x7f014c070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 --2- 172.21.15.88:0/3218298492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c156df0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 -- 172.21.15.88:0/3218298492 >> 172.21.15.88:0/3218298492 conn(0x7f014c0729d0 msgr2=0x7f014c072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:11.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 -- 172.21.15.88:0/3218298492 shutdown_connections 2024-09-21T02:04:11.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.783+0000 7f0152f4a640 1 -- 172.21.15.88:0/3218298492 wait complete. 2024-09-21T02:04:11.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.785+0000 7f0152f4a640 1 Processor -- start 2024-09-21T02:04:11.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.785+0000 7f0152f4a640 1 -- start start 2024-09-21T02:04:11.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0152f4a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c1e0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0152f4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f014c0c4f10 0x7f014c147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0152f4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c0c8a00 0x7f014c1e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0152f4a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f014c15a5b0 con 0x7f014c0c3fc0 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0152f4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f014c070ab0 con 0x7f014c0c8a00 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0152f4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f014c0c8f30 con 0x7f014c0c4f10 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0150ce6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c1e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0150ce6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c1e0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48460/0 (socket says 172.21.15.88:48460) 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f0150ce6640 1 -- 172.21.15.88:0/1456449858 learned_addr learned my addr 172.21.15.88:0/1456449858 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f014bfff640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f014c0c4f10 0x7f014c147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.786+0000 7f01514e7640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c0c8a00 0x7f014c1e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0150ce6640 1 -- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f014c0c4f10 msgr2=0x7f014c147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0150ce6640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f014c0c4f10 0x7f014c147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0150ce6640 1 -- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c0c8a00 msgr2=0x7f014c1e15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0150ce6640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c0c8a00 0x7f014c1e15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0150ce6640 1 -- 172.21.15.88:0/1456449858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c1e06a0 con 0x7f014c0c3fc0 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0150ce6640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c1e0160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f01440099c0 tx=0x7f01440039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f01497fa640 1 -- 172.21.15.88:0/1456449858 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014403e030 con 0x7f014c0c3fc0 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f01497fa640 1 -- 172.21.15.88:0/1456449858 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01440047a0 con 0x7f014c0c3fc0 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f01497fa640 1 -- 172.21.15.88:0/1456449858 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144035040 con 0x7f014c0c3fc0 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c1df8d0 con 0x7f014c0c3fc0 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.787+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c1e4920 con 0x7f014c0c3fc0 2024-09-21T02:04:11.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.788+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c1dfdf0 con 0x7f014c0c3fc0 2024-09-21T02:04:11.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.789+0000 7f01497fa640 1 -- 172.21.15.88:0/1456449858 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0144041030 con 0x7f014c0c3fc0 2024-09-21T02:04:11.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.789+0000 7f01497fa640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f012c078b90 0x7f012c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.789+0000 7f01497fa640 1 -- 172.21.15.88:0/1456449858 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f01440bb480 con 0x7f014c0c3fc0 2024-09-21T02:04:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.789+0000 7f014bfff640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f012c078b90 0x7f012c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:11.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.790+0000 7f014bfff640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f012c078b90 0x7f012c07b0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f014c0cc080 tx=0x7f013c0095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:11.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:11.794+0000 7f01497fa640 1 -- 172.21.15.88:0/1456449858 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f014c1dfdf0 con 0x7f014c0c3fc0 2024-09-21T02:04:12.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.007+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f014c072360 con 0x7f014c0c3fc0 2024-09-21T02:04:12.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.008+0000 7f01497fa640 1 -- 172.21.15.88:0/1456449858 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0144083300 con 0x7f014c0c3fc0 2024-09-21T02:04:12.009 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:12.009 INFO:teuthology.orchestra.run.smithi088.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T02:04:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f012c078b90 msgr2=0x7f012c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f012c078b90 0x7f012c07b0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f014c0cc080 tx=0x7f013c0095e0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 msgr2=0x7f014c1e0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c1e0160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f01440099c0 tx=0x7f01440039e0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 shutdown_connections 2024-09-21T02:04:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f012c078b90 0x7f012c07b0a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f014c0c8a00 0x7f014c1e15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f014c0c4f10 0x7f014c147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 --2- 172.21.15.88:0/1456449858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f014c0c3fc0 0x7f014c1e0160 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 >> 172.21.15.88:0/1456449858 conn(0x7f014c0729d0 msgr2=0x7f014c0caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:12.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 shutdown_connections 2024-09-21T02:04:12.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.010+0000 7f0152f4a640 1 -- 172.21.15.88:0/1456449858 wait complete. 2024-09-21T02:04:12.022 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T02:04:12.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:04:12.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 -- 172.21.15.88:0/2767107909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 msgr2=0x7efc000713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 --2- 172.21.15.88:0/2767107909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc000713e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc000c9b30 tx=0x7efbf802ab40 comp rx=0 tx=0).stop 2024-09-21T02:04:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 -- 172.21.15.88:0/2767107909 shutdown_connections 2024-09-21T02:04:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 --2- 172.21.15.88:0/2767107909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc000713e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 --2- 172.21.15.88:0/2767107909 >> v2:172.21.15.195:3300/0 conn(0x7efc000c8a00 0x7efc00156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 -- 172.21.15.88:0/2767107909 >> 172.21.15.88:0/2767107909 conn(0x7efc000729d0 msgr2=0x7efc00072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 -- 172.21.15.88:0/2767107909 shutdown_connections 2024-09-21T02:04:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.135+0000 7efc08377640 1 -- 172.21.15.88:0/2767107909 wait complete. 2024-09-21T02:04:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.137+0000 7efc08377640 1 Processor -- start 2024-09-21T02:04:12.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.137+0000 7efc08377640 1 -- start start 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc08377640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc0006f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc08377640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc000c3fc0 0x7efc0014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc08377640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc000c8a00 0x7efc0014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc08377640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc00071920 con 0x7efc00070fd0 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc08377640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efc00070ab0 con 0x7efc000c8a00 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc08377640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efc0015a5b0 con 0x7efc000c3fc0 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc0006f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc0006f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48484/0 (socket says 172.21.15.88:48484) 2024-09-21T02:04:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 -- 172.21.15.88:0/3901357470 learned_addr learned my addr 172.21.15.88:0/3901357470 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06914640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc000c8a00 0x7efc0014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc05912640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc000c3fc0 0x7efc0014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 -- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc000c3fc0 msgr2=0x7efc0014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc000c3fc0 0x7efc0014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 -- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc000c8a00 msgr2=0x7efc0014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc000c8a00 0x7efc0014aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 -- 172.21.15.88:0/3901357470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0006f6b0 con 0x7efc00070fd0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.138+0000 7efc06113640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc0006f170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00c3d0 tx=0x7efbfc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.139+0000 7efbf6ffd640 1 -- 172.21.15.88:0/3901357470 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc017070 con 0x7efc00070fd0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.139+0000 7efbf6ffd640 1 -- 172.21.15.88:0/3901357470 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbfc0053c0 con 0x7efc00070fd0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.139+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc001556a0 con 0x7efc00070fd0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.139+0000 7efbf6ffd640 1 -- 172.21.15.88:0/3901357470 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc003850 con 0x7efc00070fd0 2024-09-21T02:04:12.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.139+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0006de70 con 0x7efc00070fd0 2024-09-21T02:04:12.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.140+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc000c8f30 con 0x7efc00070fd0 2024-09-21T02:04:12.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.140+0000 7efbf6ffd640 1 -- 172.21.15.88:0/3901357470 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efbfc01a070 con 0x7efc00070fd0 2024-09-21T02:04:12.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.141+0000 7efbf6ffd640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efbe4078b90 0x7efbe407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.141+0000 7efbf6ffd640 1 -- 172.21.15.88:0/3901357470 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efbfc099f40 con 0x7efc00070fd0 2024-09-21T02:04:12.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.141+0000 7efc05912640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efbe4078b90 0x7efbe407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.141+0000 7efc05912640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efbe4078b90 0x7efbe407b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc00158090 tx=0x7efbf0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:12.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.145+0000 7efbf6ffd640 1 -- 172.21.15.88:0/3901357470 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efbfc061e50 con 0x7efc00070fd0 2024-09-21T02:04:12.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.360+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc00072de0 con 0x7efc00070fd0 2024-09-21T02:04:12.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.360+0000 7efbf6ffd640 1 -- 172.21.15.88:0/3901357470 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6905 (secure 0 0 0) 0x7efbfc066880 con 0x7efc00070fd0 2024-09-21T02:04:12.362 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:12.362 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":12,"btime":"2024-09-21T02:04:10:127884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:10.127864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4401,4399]},"id":1}]} 2024-09-21T02:04:12.362 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 12 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.362+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efbe4078b90 msgr2=0x7efbe407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efbe4078b90 0x7efbe407b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc00158090 tx=0x7efbf0005f90 comp rx=0 tx=0).stop 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 msgr2=0x7efc0006f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc0006f170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00c3d0 tx=0x7efbfc00c890 comp rx=0 tx=0).stop 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 shutdown_connections 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efbe4078b90 0x7efbe407b0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc000c8a00 0x7efc0014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc000c3fc0 0x7efc0014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 --2- 172.21.15.88:0/3901357470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc00070fd0 0x7efc0006f170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 >> 172.21.15.88:0/3901357470 conn(0x7efc000729d0 msgr2=0x7efc00158600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:12.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 shutdown_connections 2024-09-21T02:04:12.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.363+0000 7efc08377640 1 -- 172.21.15.88:0/3901357470 wait complete. 2024-09-21T02:04:12.375 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:04:12.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 -- 172.21.15.88:0/2446135368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c073d70 msgr2=0x7f242c159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 --2- 172.21.15.88:0/2446135368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c073d70 0x7f242c159180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f241c005fd0 tx=0x7f241c02aa90 comp rx=0 tx=0).stop 2024-09-21T02:04:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 -- 172.21.15.88:0/2446135368 shutdown_connections 2024-09-21T02:04:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 --2- 172.21.15.88:0/2446135368 >> v2:172.21.15.195:3300/0 conn(0x7f242c0c2e60 0x7f242c071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 --2- 172.21.15.88:0/2446135368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c073d70 0x7f242c159180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 -- 172.21.15.88:0/2446135368 >> 172.21.15.88:0/2446135368 conn(0x7f242c0ce440 msgr2=0x7f242c0ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 -- 172.21.15.88:0/2446135368 shutdown_connections 2024-09-21T02:04:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.486+0000 7f243147e640 1 -- 172.21.15.88:0/2446135368 wait complete. 2024-09-21T02:04:12.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.488+0000 7f243147e640 1 Processor -- start 2024-09-21T02:04:12.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.488+0000 7f243147e640 1 -- start start 2024-09-21T02:04:12.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f243147e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f242c073d70 0x7f242c14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f243147e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c0c2e60 0x7f242c1f0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f243147e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f242c0c87b0 0x7f242c1455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f243147e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0cb740 con 0x7f242c0c87b0 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f243147e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0cb5c0 con 0x7f242c073d70 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f243147e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f242c071c70 con 0x7f242c0c2e60 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f242b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f242c0c87b0 0x7f242c1455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f242b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f242c0c87b0 0x7f242c1455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48492/0 (socket says 172.21.15.88:48492) 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f242b7fe640 1 -- 172.21.15.88:0/2185896882 learned_addr learned my addr 172.21.15.88:0/2185896882 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f242a7fc640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c0c2e60 0x7f242c1f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.489+0000 7f242affd640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f242c073d70 0x7f242c14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f242b7fe640 1 -- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c0c2e60 msgr2=0x7f242c1f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f242b7fe640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c0c2e60 0x7f242c1f0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f242b7fe640 1 -- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f242c073d70 msgr2=0x7f242c14bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f242b7fe640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f242c073d70 0x7f242c14bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f242b7fe640 1 -- 172.21.15.88:0/2185896882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c0c60f0 con 0x7f242c0c87b0 2024-09-21T02:04:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f242b7fe640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f242c0c87b0 0x7f242c1455d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f242400c3f0 tx=0x7f242400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f24137fe640 1 -- 172.21.15.88:0/2185896882 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424017070 con 0x7f242c0c87b0 2024-09-21T02:04:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f24137fe640 1 -- 172.21.15.88:0/2185896882 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24240053c0 con 0x7f242c0c87b0 2024-09-21T02:04:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c14c4c0 con 0x7f242c0c87b0 2024-09-21T02:04:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f24137fe640 1 -- 172.21.15.88:0/2185896882 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424003850 con 0x7f242c0c87b0 2024-09-21T02:04:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.490+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c149620 con 0x7f242c0c87b0 2024-09-21T02:04:12.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.491+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c0c3f40 con 0x7f242c0c87b0 2024-09-21T02:04:12.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.492+0000 7f24137fe640 1 -- 172.21.15.88:0/2185896882 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f242401a070 con 0x7f242c0c87b0 2024-09-21T02:04:12.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.492+0000 7f24137fe640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2418078bc0 0x7f241807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.492+0000 7f24137fe640 1 -- 172.21.15.88:0/2185896882 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2424099f80 con 0x7f242c0c87b0 2024-09-21T02:04:12.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.492+0000 7f242affd640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2418078bc0 0x7f241807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.493+0000 7f242affd640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2418078bc0 0x7f241807b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f241c035000 tx=0x7f241c038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:12.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.497+0000 7f24137fe640 1 -- 172.21.15.88:0/2185896882 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f242c0c3f40 con 0x7f242c0c87b0 2024-09-21T02:04:12.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.708+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f242c0b3c50 con 0x7f242c0c87b0 2024-09-21T02:04:12.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.709+0000 7f24137fe640 1 -- 172.21.15.88:0/2185896882 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6905 (secure 0 0 0) 0x7f2424061e40 con 0x7f242c0c87b0 2024-09-21T02:04:12.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:12.710 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":12,"btime":"2024-09-21T02:04:10:127884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:10.127864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4401,4399]},"id":1}]} 2024-09-21T02:04:12.710 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 12 2024-09-21T02:04:12.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2418078bc0 msgr2=0x7f241807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2418078bc0 0x7f241807b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f241c035000 tx=0x7f241c038040 comp rx=0 tx=0).stop 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f242c0c87b0 msgr2=0x7f242c1455d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f242c0c87b0 0x7f242c1455d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f242400c3f0 tx=0x7f242400c8b0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 shutdown_connections 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2418078bc0 0x7f241807b0d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f242c0c87b0 0x7f242c1455d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242c0c2e60 0x7f242c1f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 --2- 172.21.15.88:0/2185896882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f242c073d70 0x7f242c14bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 >> 172.21.15.88:0/2185896882 conn(0x7f242c0ce440 msgr2=0x7f242c148a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 shutdown_connections 2024-09-21T02:04:12.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.711+0000 7f243147e640 1 -- 172.21.15.88:0/2185896882 wait complete. 2024-09-21T02:04:12.724 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:04:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/150116603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c41a0 msgr2=0x7f6c640c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/150116603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c41a0 0x7f6c640c45b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009830 tx=0x7f6c5c02a4a0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/150116603 shutdown_connections 2024-09-21T02:04:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/150116603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c41a0 0x7f6c640c45b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/150116603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c64153320 0x7f6c64153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/150116603 >> 172.21.15.88:0/150116603 conn(0x7f6c64073210 msgr2=0x7f6c640cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/150116603 shutdown_connections 2024-09-21T02:04:12.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.834+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/150116603 wait complete. 2024-09-21T02:04:12.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.836+0000 7f6c6b1e6640 1 Processor -- start 2024-09-21T02:04:12.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.836+0000 7f6c6b1e6640 1 -- start start 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.837+0000 7f6c6b1e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c640c41a0 0x7f6c640c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.837+0000 7f6c6b1e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c99a0 0x7f6c64158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c6b1e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c64153320 0x7f6c6406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c6b1e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64070ae0 con 0x7f6c640c41a0 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c6b1e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64072000 con 0x7f6c640c99a0 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c6b1e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6c64070dc0 con 0x7f6c64153320 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c640c41a0 0x7f6c640c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c640c41a0 0x7f6c640c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48504/0 (socket says 172.21.15.88:48504) 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 -- 172.21.15.88:0/3576479930 learned_addr learned my addr 172.21.15.88:0/3576479930 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c63fff640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c99a0 0x7f6c64158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c69783640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c64153320 0x7f6c6406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 -- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c64153320 msgr2=0x7f6c6406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c64153320 0x7f6c6406f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 -- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c99a0 msgr2=0x7f6c64158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c99a0 0x7f6c64158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 -- 172.21.15.88:0/3576479930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c641f2050 con 0x7f6c640c41a0 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c69783640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c64153320 0x7f6c6406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c68f82640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c640c41a0 0x7f6c640c8650 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c530 tx=0x7f6c5400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c617fa640 1 -- 172.21.15.88:0/3576479930 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54016020 con 0x7f6c640c41a0 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c640c9070 con 0x7f6c640c41a0 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c617fa640 1 -- 172.21.15.88:0/3576479930 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c54012940 con 0x7f6c640c41a0 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c617fa640 1 -- 172.21.15.88:0/3576479930 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54012d90 con 0x7f6c640c41a0 2024-09-21T02:04:12.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.838+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c64148a10 con 0x7f6c640c41a0 2024-09-21T02:04:12.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.839+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c641f2220 con 0x7f6c640c41a0 2024-09-21T02:04:12.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.840+0000 7f6c617fa640 1 -- 172.21.15.88:0/3576479930 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c54019020 con 0x7f6c640c41a0 2024-09-21T02:04:12.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.841+0000 7f6c617fa640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c50078bf0 0x7f6c5007b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:12.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.841+0000 7f6c617fa640 1 -- 172.21.15.88:0/3576479930 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6c5409aa50 con 0x7f6c640c41a0 2024-09-21T02:04:12.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.841+0000 7f6c63fff640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c50078bf0 0x7f6c5007b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:12.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.841+0000 7f6c63fff640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c50078bf0 0x7f6c5007b100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c64159cc0 tx=0x7f6c58009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:12.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:12.845+0000 7f6c617fa640 1 -- 172.21.15.88:0/3576479930 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c54020030 con 0x7f6c640c41a0 2024-09-21T02:04:13.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.022+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c640ca8c0 con 0x7f6c640c41a0 2024-09-21T02:04:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.023+0000 7f6c617fa640 1 -- 172.21.15.88:0/3576479930 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f6c54062e10 con 0x7f6c640c41a0 2024-09-21T02:04:13.024 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:13.024 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":21,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:59.066086+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c50078bf0 msgr2=0x7f6c5007b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c50078bf0 0x7f6c5007b100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c64159cc0 tx=0x7f6c58009500 comp rx=0 tx=0).stop 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c640c41a0 msgr2=0x7f6c640c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c640c41a0 0x7f6c640c8650 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c530 tx=0x7f6c5400c9f0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 shutdown_connections 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c50078bf0 0x7f6c5007b100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c64153320 0x7f6c6406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c640c99a0 0x7f6c64158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 --2- 172.21.15.88:0/3576479930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c640c41a0 0x7f6c640c8650 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 >> 172.21.15.88:0/3576479930 conn(0x7f6c64073210 msgr2=0x7f6c64150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 shutdown_connections 2024-09-21T02:04:13.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.025+0000 7f6c6b1e6640 1 -- 172.21.15.88:0/3576479930 wait complete. 2024-09-21T02:04:13.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:04:13.047 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-09-21T02:04:13.047+0000 7fd9fc4cd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:04:13.047 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-09-21T02:04:13.047+0000 7fd9fc4cd640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 -- 172.21.15.88:0/3900022455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40cad60 msgr2=0x7f1ba406cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/3900022455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40cad60 0x7f1ba406cd60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40754d0 tx=0x7f1b9802a610 comp rx=0 tx=0).stop 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 -- 172.21.15.88:0/3900022455 shutdown_connections 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/3900022455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40cad60 0x7f1ba406cd60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/3900022455 >> v2:172.21.15.195:3300/0 conn(0x7f1ba40ccd00 0x7f1ba415f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 -- 172.21.15.88:0/3900022455 >> 172.21.15.88:0/3900022455 conn(0x7f1ba40cd9f0 msgr2=0x7f1ba40cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 -- 172.21.15.88:0/3900022455 shutdown_connections 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.151+0000 7f1ba92fa640 1 -- 172.21.15.88:0/3900022455 wait complete. 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.153+0000 7f1ba92fa640 1 Processor -- start 2024-09-21T02:04:13.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.153+0000 7f1ba92fa640 1 -- start start 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba92fa640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ba40cad60 0x7f1ba41f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba92fa640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ba40cb170 0x7f1ba41f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba92fa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40ccd00 0x7f1ba415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba92fa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba406dcd0 con 0x7f1ba40ccd00 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba92fa640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba415c640 con 0x7f1ba40cad60 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba92fa640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba406de50 con 0x7f1ba40cb170 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40ccd00 0x7f1ba415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40ccd00 0x7f1ba415dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48530/0 (socket says 172.21.15.88:48530) 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 -- 172.21.15.88:0/1442035812 learned_addr learned my addr 172.21.15.88:0/1442035812 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba2ffd640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ba40cad60 0x7f1ba41f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 -- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ba40cb170 msgr2=0x7f1ba41f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba27fc640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ba40cb170 0x7f1ba41f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ba40cb170 0x7f1ba41f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 -- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ba40cad60 msgr2=0x7f1ba41f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ba40cad60 0x7f1ba41f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 -- 172.21.15.88:0/1442035812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41f3cb0 con 0x7f1ba40ccd00 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba2ffd640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ba40cad60 0x7f1ba41f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:13.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.154+0000 7f1ba37fe640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40ccd00 0x7f1ba415dcd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00b410 tx=0x7f1b9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.155+0000 7f1b8b7fe640 1 -- 172.21.15.88:0/1442035812 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c015030 con 0x7f1ba40ccd00 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.155+0000 7f1b8b7fe640 1 -- 172.21.15.88:0/1442035812 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b9c004020 con 0x7f1ba40ccd00 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.155+0000 7f1b8b7fe640 1 -- 172.21.15.88:0/1442035812 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c011630 con 0x7f1ba40ccd00 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.155+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41f2370 con 0x7f1ba40ccd00 2024-09-21T02:04:13.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.155+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41f2930 con 0x7f1ba40ccd00 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.156+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba41f20c0 con 0x7f1ba40ccd00 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.156+0000 7f1b8b7fe640 1 -- 172.21.15.88:0/1442035812 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b9c018070 con 0x7f1ba40ccd00 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.157+0000 7f1b8b7fe640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1b90078bc0 0x7f1b9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.157+0000 7f1b8b7fe640 1 -- 172.21.15.88:0/1442035812 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1b9c09a730 con 0x7f1ba40ccd00 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.157+0000 7f1ba2ffd640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1b90078bc0 0x7f1b9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:13.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.158+0000 7f1ba2ffd640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1b90078bc0 0x7f1b9007b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005c40 tx=0x7f1b94005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:13.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.161+0000 7f1b8b7fe640 1 -- 172.21.15.88:0/1442035812 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b9c062c00 con 0x7f1ba40ccd00 2024-09-21T02:04:13.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.376+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ba4155f30 con 0x7f1ba40ccd00 2024-09-21T02:04:13.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.377+0000 7f1b8b7fe640 1 -- 172.21.15.88:0/1442035812 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6905 (secure 0 0 0) 0x7f1b9c062500 con 0x7f1ba40ccd00 2024-09-21T02:04:13.378 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:13.378 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":12,"btime":"2024-09-21T02:04:10:127884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:10.127864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4401,4399]},"id":1}]} 2024-09-21T02:04:13.378 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 12 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1b90078bc0 msgr2=0x7f1b9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1b90078bc0 0x7f1b9007b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005c40 tx=0x7f1b94005f90 comp rx=0 tx=0).stop 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40ccd00 msgr2=0x7f1ba415dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40ccd00 0x7f1ba415dcd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00b410 tx=0x7f1b9c00b8d0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 shutdown_connections 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1b90078bc0 0x7f1b9007b0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ba40ccd00 0x7f1ba415dcd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ba40cb170 0x7f1ba41f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 --2- 172.21.15.88:0/1442035812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ba40cad60 0x7f1ba41f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:13.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 >> 172.21.15.88:0/1442035812 conn(0x7f1ba40cd9f0 msgr2=0x7f1ba41554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:13.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 shutdown_connections 2024-09-21T02:04:13.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:13.379+0000 7f1ba92fa640 1 -- 172.21.15.88:0/1442035812 wait complete. 2024-09-21T02:04:13.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:03:57.430092+0000', 'modified': '2024-09-21T02:04:10.127864+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4401, 'mds_1': 4399, 'mds_2': 4398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4398': {'gid': 4398, 'name': 'f', 'rank': 2, 'incarnation': 12, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.195:6837/778249012', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 778249012}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 778249012}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4399': {'gid': 4399, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.88:6837/3330183943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 3330183943}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 3330183943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4401': {'gid': 4401, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.195:6839/3087611678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 3087611678}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 3087611678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4401, 'qdb_cluster': [4401, 4399]} 2024-09-21T02:04:13.995 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4398:up:creating 2024-09-21T02:04:14.996 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:04:15.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 -- 172.21.15.88:0/556446817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2550 msgr2=0x7fab6806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:15.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 --2- 172.21.15.88:0/556446817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2550 0x7fab6806cd60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab680ca400 tx=0x7fab6002aac0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 -- 172.21.15.88:0/556446817 shutdown_connections 2024-09-21T02:04:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 --2- 172.21.15.88:0/556446817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2550 0x7fab6806cd60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 --2- 172.21.15.88:0/556446817 >> v2:172.21.15.195:3300/0 conn(0x7fab68151f50 0x7fab680b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 -- 172.21.15.88:0/556446817 >> 172.21.15.88:0/556446817 conn(0x7fab68070c30 msgr2=0x7fab68071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 -- 172.21.15.88:0/556446817 shutdown_connections 2024-09-21T02:04:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.121+0000 7fab6e96a640 1 -- 172.21.15.88:0/556446817 wait complete. 2024-09-21T02:04:15.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.123+0000 7fab6e96a640 1 Processor -- start 2024-09-21T02:04:15.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.123+0000 7fab6e96a640 1 -- start start 2024-09-21T02:04:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab6e96a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2140 0x7fab680c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab6e96a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab68151f50 0x7fab68153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab6e96a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6814efa0 0x7fab68154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab6e96a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fab680c8330 con 0x7fab680c2140 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab6e96a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fab680734d0 con 0x7fab68151f50 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab6e96a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fab68157e20 con 0x7fab6814efa0 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab67fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2140 0x7fab680c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab67fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2140 0x7fab680c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48542/0 (socket says 172.21.15.88:48542) 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab67fff640 1 -- 172.21.15.88:0/1477641353 learned_addr learned my addr 172.21.15.88:0/1477641353 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.124+0000 7fab677fe640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab68151f50 0x7fab68153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab6cf07640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6814efa0 0x7fab68154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab67fff640 1 -- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6814efa0 msgr2=0x7fab68154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab67fff640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6814efa0 0x7fab68154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab67fff640 1 -- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab68151f50 msgr2=0x7fab68153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab67fff640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab68151f50 0x7fab68153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab67fff640 1 -- 172.21.15.88:0/1477641353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab680c4ba0 con 0x7fab680c2140 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab67fff640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2140 0x7fab680c4660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab5800d770 tx=0x7fab5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab64ff9640 1 -- 172.21.15.88:0/1477641353 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58017070 con 0x7fab680c2140 2024-09-21T02:04:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab64ff9640 1 -- 172.21.15.88:0/1477641353 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab58009d20 con 0x7fab680c2140 2024-09-21T02:04:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab64ff9640 1 -- 172.21.15.88:0/1477641353 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab580106c0 con 0x7fab680c2140 2024-09-21T02:04:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab681ef240 con 0x7fab680c2140 2024-09-21T02:04:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.125+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab681ef4e0 con 0x7fab680c2140 2024-09-21T02:04:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.127+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab680c2550 con 0x7fab680c2140 2024-09-21T02:04:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.127+0000 7fab64ff9640 1 -- 172.21.15.88:0/1477641353 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fab5801a070 con 0x7fab680c2140 2024-09-21T02:04:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.128+0000 7fab64ff9640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fab54078bc0 0x7fab5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.128+0000 7fab64ff9640 1 -- 172.21.15.88:0/1477641353 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fab5809ad90 con 0x7fab680c2140 2024-09-21T02:04:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.128+0000 7fab677fe640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fab54078bc0 0x7fab5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.129+0000 7fab677fe640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fab54078bc0 0x7fab5407b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab68147330 tx=0x7fab5c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:15.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.133+0000 7fab64ff9640 1 -- 172.21.15.88:0/1477641353 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fab58062c10 con 0x7fab680c2140 2024-09-21T02:04:15.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.350+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab6814f3b0 con 0x7fab680c2140 2024-09-21T02:04:15.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.350+0000 7fab64ff9640 1 -- 172.21.15.88:0/1477641353 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6905 (secure 0 0 0) 0x7fab58067640 con 0x7fab680c2140 2024-09-21T02:04:15.352 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:15.352 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":12,"btime":"2024-09-21T02:04:10:127884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:10.127864+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4401,4399]},"id":1}]} 2024-09-21T02:04:15.352 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 12 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fab54078bc0 msgr2=0x7fab5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fab54078bc0 0x7fab5407b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab68147330 tx=0x7fab5c009040 comp rx=0 tx=0).stop 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2140 msgr2=0x7fab680c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2140 0x7fab680c4660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab5800d770 tx=0x7fab5800dc30 comp rx=0 tx=0).stop 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 shutdown_connections 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fab54078bc0 0x7fab5407b0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6814efa0 0x7fab68154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab68151f50 0x7fab68153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 --2- 172.21.15.88:0/1477641353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab680c2140 0x7fab680c4660 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 >> 172.21.15.88:0/1477641353 conn(0x7fab68070c30 msgr2=0x7fab681476c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 shutdown_connections 2024-09-21T02:04:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:15.353+0000 7fab6e96a640 1 -- 172.21.15.88:0/1477641353 wait complete. 2024-09-21T02:04:15.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:03:57.430092+0000', 'modified': '2024-09-21T02:04:10.127864+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4401, 'mds_1': 4399, 'mds_2': 4398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4398': {'gid': 4398, 'name': 'f', 'rank': 2, 'incarnation': 12, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.195:6837/778249012', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 778249012}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 778249012}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4399': {'gid': 4399, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.88:6837/3330183943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 3330183943}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 3330183943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4401': {'gid': 4401, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.195:6839/3087611678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 3087611678}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 3087611678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4401, 'qdb_cluster': [4401, 4399]} 2024-09-21T02:04:15.969 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4398:up:creating 2024-09-21T02:04:16.970 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:04:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.095+0000 7f47432c5640 1 -- 172.21.15.88:0/3062378575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4728004800 msgr2=0x7f4728004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.095+0000 7f47432c5640 1 --2- 172.21.15.88:0/3062378575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4728004800 0x7f4728004c10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f472c0016c0 tx=0x7f4728005830 comp rx=0 tx=0).stop 2024-09-21T02:04:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.095+0000 7f47432c5640 1 -- 172.21.15.88:0/3062378575 shutdown_connections 2024-09-21T02:04:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.095+0000 7f47432c5640 1 --2- 172.21.15.88:0/3062378575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4728004800 0x7f4728004c10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.095+0000 7f47432c5640 1 -- 172.21.15.88:0/3062378575 >> 172.21.15.88:0/3062378575 conn(0x7f473c0cd9f0 msgr2=0x7f473c0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.095+0000 7f47432c5640 1 -- 172.21.15.88:0/3062378575 shutdown_connections 2024-09-21T02:04:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.095+0000 7f47432c5640 1 -- 172.21.15.88:0/3062378575 wait complete. 2024-09-21T02:04:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.096+0000 7f47432c5640 1 Processor -- start 2024-09-21T02:04:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.097+0000 7f47432c5640 1 -- start start 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f47432c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4728004800 0x7f473c1e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f47432c5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c0763e0 0x7f473c1e7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f47432c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c0767f0 0x7f473c1f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f47432c5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f473c15fd80 con 0x7f473c0763e0 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f47432c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f473c15c3b0 con 0x7f473c0767f0 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f47432c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f473c06dcd0 con 0x7f4728004800 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c0763e0 0x7f473c1e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c0763e0 0x7f473c1e7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48556/0 (socket says 172.21.15.88:48556) 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 -- 172.21.15.88:0/1243877860 learned_addr learned my addr 172.21.15.88:0/1243877860 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:17.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4741061640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4728004800 0x7f473c1e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4741862640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c0767f0 0x7f473c1f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 -- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4728004800 msgr2=0x7f473c1e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4728004800 0x7f473c1e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 -- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c0767f0 msgr2=0x7f473c1f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c0767f0 0x7f473c1f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 -- 172.21.15.88:0/1243877860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c1e6640 con 0x7f473c0763e0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4741862640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c0767f0 0x7f473c1f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4740860640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c0763e0 0x7f473c1e7000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f472c0017e0 tx=0x7f472c0063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.098+0000 7f4731ffb640 1 -- 172.21.15.88:0/1243877860 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c042070 con 0x7f473c0763e0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.099+0000 7f4731ffb640 1 -- 172.21.15.88:0/1243877860 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f472c039070 con 0x7f473c0763e0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.099+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c0c78d0 con 0x7f473c0763e0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.099+0000 7f4731ffb640 1 -- 172.21.15.88:0/1243877860 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c003d90 con 0x7f473c0763e0 2024-09-21T02:04:17.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.099+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c0d0850 con 0x7f473c0763e0 2024-09-21T02:04:17.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.100+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c0cc030 con 0x7f473c0763e0 2024-09-21T02:04:17.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.100+0000 7f4731ffb640 1 -- 172.21.15.88:0/1243877860 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f472c03b070 con 0x7f473c0763e0 2024-09-21T02:04:17.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.101+0000 7f4731ffb640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4720078b90 0x7f472007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:17.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.101+0000 7f4731ffb640 1 -- 172.21.15.88:0/1243877860 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f472c0ba2d0 con 0x7f473c0763e0 2024-09-21T02:04:17.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.101+0000 7f4741061640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4720078b90 0x7f472007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:17.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.102+0000 7f4741061640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4720078b90 0x7f472007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4738004b00 tx=0x7f4738006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:17.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.105+0000 7f4731ffb640 1 -- 172.21.15.88:0/1243877860 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f472c0821e0 con 0x7f473c0763e0 2024-09-21T02:04:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.320+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f473c155f30 con 0x7f473c0763e0 2024-09-21T02:04:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.321+0000 7f4731ffb640 1 -- 172.21.15.88:0/1243877860 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6908 (secure 0 0 0) 0x7f472c086c10 con 0x7f473c0763e0 2024-09-21T02:04:17.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:17.322 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":13,"btime":"2024-09-21T02:04:16:141092+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:16.141080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4398,4399,4401]},"id":1}]} 2024-09-21T02:04:17.322 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 13 2024-09-21T02:04:17.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4720078b90 msgr2=0x7f472007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4720078b90 0x7f472007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4738004b00 tx=0x7f4738006040 comp rx=0 tx=0).stop 2024-09-21T02:04:17.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c0763e0 msgr2=0x7f473c1e7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:17.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c0763e0 0x7f473c1e7000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f472c0017e0 tx=0x7f472c0063a0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 shutdown_connections 2024-09-21T02:04:17.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4720078b90 0x7f472007b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f473c0767f0 0x7f473c1f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c0763e0 0x7f473c1e7000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 --2- 172.21.15.88:0/1243877860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4728004800 0x7f473c1e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 >> 172.21.15.88:0/1243877860 conn(0x7f473c0cd9f0 msgr2=0x7f473c1555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 shutdown_connections 2024-09-21T02:04:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:17.323+0000 7f47432c5640 1 -- 172.21.15.88:0/1243877860 wait complete. 2024-09-21T02:04:17.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:03:57.430092+0000', 'modified': '2024-09-21T02:04:16.141080+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4401, 'mds_1': 4399, 'mds_2': 4398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4398': {'gid': 4398, 'name': 'f', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.195:6837/778249012', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 778249012}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 778249012}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4399': {'gid': 4399, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.88:6837/3330183943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 3330183943}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 3330183943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4401': {'gid': 4401, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.195:6839/3087611678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 3087611678}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 3087611678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4401, 'qdb_cluster': [4398, 4399, 4401]} 2024-09-21T02:04:17.940 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T02:04:17.940 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-09-21T02:04:18.048 INFO:tasks.ceph.mds.d.smithi195.stderr:2024-09-21T02:04:18.047+0000 7fd9fc4cd640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T02:04:18.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 -- 172.21.15.88:0/1250162448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 msgr2=0x7f561c0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:18.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 --2- 172.21.15.88:0/1250162448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c0cafc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5610005fd0 tx=0x7f561002a5d0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 -- 172.21.15.88:0/1250162448 shutdown_connections 2024-09-21T02:04:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 --2- 172.21.15.88:0/1250162448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c0cafc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 --2- 172.21.15.88:0/1250162448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561c15b980 0x7f561c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 -- 172.21.15.88:0/1250162448 >> 172.21.15.88:0/1250162448 conn(0x7f561c0cda90 msgr2=0x7f561c0cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:18.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 -- 172.21.15.88:0/1250162448 shutdown_connections 2024-09-21T02:04:18.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.053+0000 7f562175f640 1 -- 172.21.15.88:0/1250162448 wait complete. 2024-09-21T02:04:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.054+0000 7f562175f640 1 Processor -- start 2024-09-21T02:04:18.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.055+0000 7f562175f640 1 -- start start 2024-09-21T02:04:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f562175f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561c0b51f0 0x7f561c14ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f562175f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f562175f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f561c15b980 0x7f561c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f562175f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f561c0b46f0 con 0x7f561c0cb990 2024-09-21T02:04:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f562175f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f561c0b4410 con 0x7f561c15b980 2024-09-21T02:04:18.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f562175f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f561c06d470 con 0x7f561c0b51f0 2024-09-21T02:04:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c14f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49528/0 (socket says 172.21.15.88:49528) 2024-09-21T02:04:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 -- 172.21.15.88:0/2159322585 learned_addr learned my addr 172.21.15.88:0/2159322585 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 -- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561c0b51f0 msgr2=0x7f561c14ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561affd640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561c0b51f0 0x7f561c14ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561b7fe640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f561c15b980 0x7f561c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561c0b51f0 0x7f561c14ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 -- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f561c15b980 msgr2=0x7f561c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f561c15b980 0x7f561c152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 -- 172.21.15.88:0/2159322585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c1e44e0 con 0x7f561c0cb990 2024-09-21T02:04:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f561a7fc640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c14f4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5610000c00 tx=0x7f56100038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:18.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f56037fe640 1 -- 172.21.15.88:0/2159322585 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561003e030 con 0x7f561c0cb990 2024-09-21T02:04:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f56037fe640 1 -- 172.21.15.88:0/2159322585 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5610003ca0 con 0x7f561c0cb990 2024-09-21T02:04:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f56037fe640 1 -- 172.21.15.88:0/2159322585 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610035040 con 0x7f561c0cb990 2024-09-21T02:04:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.056+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c14d7a0 con 0x7f561c0cb990 2024-09-21T02:04:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.057+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c1e4a80 con 0x7f561c0cb990 2024-09-21T02:04:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.058+0000 7f56037fe640 1 -- 172.21.15.88:0/2159322585 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5610041030 con 0x7f561c0cb990 2024-09-21T02:04:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.059+0000 7f56037fe640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5608078bc0 0x7f560807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.059+0000 7f56037fe640 1 -- 172.21.15.88:0/2159322585 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f56100bb3c0 con 0x7f561c0cb990 2024-09-21T02:04:18.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.059+0000 7f561affd640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5608078bc0 0x7f560807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.060+0000 7f561affd640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5608078bc0 0x7f560807b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f560c005fd0 tx=0x7f560c005f40 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:18.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.060+0000 7f56286c9740 1 Processor -- start 2024-09-21T02:04:18.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.060+0000 7f56286c9740 1 -- start start 2024-09-21T02:04:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bc61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b7c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f56286c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b81b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f56286c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ecf0af4820 con 0x55ecf07b9590 2024-09-21T02:04:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecf0ae3c50 con 0x55ecf07da270 2024-09-21T02:04:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ecf0ae7a60 con 0x55ecf07b13b0 2024-09-21T02:04:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b81b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f561affd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bc61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b81b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49530/0 (socket says 172.21.15.88:49530) 2024-09-21T02:04:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f561a7fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b7c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.061+0000 7f561b7fe640 1 -- 172.21.15.88:0/459337702 learned_addr learned my addr 172.21.15.88:0/459337702 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f561b7fe640 1 -- 172.21.15.88:0/459337702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 msgr2=0x55ecf0b7c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f561b7fe640 1 --2- 172.21.15.88:0/459337702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b7c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f561b7fe640 1 -- 172.21.15.88:0/459337702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 msgr2=0x55ecf0bc61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f561b7fe640 1 --2- 172.21.15.88:0/459337702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bc61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f561b7fe640 1 -- 172.21.15.88:0/459337702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf0845850 con 0x55ecf07b9590 2024-09-21T02:04:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f561b7fe640 1 --2- 172.21.15.88:0/459337702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b81b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5614001590 tx=0x7f5614001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e21b22673e8583 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f5600ff9640 1 -- 172.21.15.88:0/459337702 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614018070 con 0x55ecf07b9590 2024-09-21T02:04:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f5600ff9640 1 -- 172.21.15.88:0/459337702 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56140053c0 con 0x55ecf07b9590 2024-09-21T02:04:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.062+0000 7f5600ff9640 1 -- 172.21.15.88:0/459337702 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561400d990 con 0x55ecf07b9590 2024-09-21T02:04:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 -- 172.21.15.88:0/459337702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 msgr2=0x55ecf0b81b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 --2- 172.21.15.88:0/459337702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b81b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5614001590 tx=0x7f5614001a50 comp rx=0 tx=0).stop 2024-09-21T02:04:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 -- 172.21.15.88:0/459337702 shutdown_connections 2024-09-21T02:04:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 --2- 172.21.15.88:0/459337702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b81b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 --2- 172.21.15.88:0/459337702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b7c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 --2- 172.21.15.88:0/459337702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bc61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 -- 172.21.15.88:0/459337702 >> 172.21.15.88:0/459337702 conn(0x55ecf07ce3c0 msgr2=0x55ecf0bc5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 -- 172.21.15.88:0/459337702 shutdown_connections 2024-09-21T02:04:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.063+0000 7f56286c9740 1 -- 172.21.15.88:0/459337702 wait complete. 2024-09-21T02:04:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.064+0000 7f56286c9740 1 -- start start 2024-09-21T02:04:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.064+0000 7f56286c9740 1 Processor -- start 2024-09-21T02:04:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0bb90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bb2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f56286c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bc0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f56286c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ecf0854620 con 0x55ecf07da270 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecf0a895f0 con 0x55ecf07b13b0 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ecf0bb9ae0 con 0x55ecf07b9590 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f56286c9740 10 client.0 _ll_register_callbacks cb 0x55ecf0a8d0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bc0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bc0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49540/0 (socket says 172.21.15.88:49540) 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 -- 172.21.15.88:0/3827683933 learned_addr learned my addr 172.21.15.88:0/3827683933 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 -- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 msgr2=0x55ecf0bb2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561affd640 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0bb90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561a7fc640 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bb2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bb2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 -- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 msgr2=0x55ecf0bb90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0bb90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.065+0000 7f561b7fe640 1 -- 172.21.15.88:0/3827683933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf08bc6b0 con 0x55ecf07da270 2024-09-21T02:04:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f561b7fe640 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bc0500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5614009010 tx=0x7f561400dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8872786608978e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f55f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f55f9ffb640 1 -- 172.21.15.88:0/3827683933 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614018050 con 0x55ecf07da270 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56140053c0 con 0x55ecf07da270 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f55f9ffb640 1 -- 172.21.15.88:0/3827683933 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5614007030 con 0x55ecf07da270 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f55f9ffb640 1 -- 172.21.15.88:0/3827683933 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f561402b050 con 0x55ecf07da270 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 10 client.4734 fetch_fsmap learned FSMap version 13 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x55ecf0bb3190 con 0x55ecf07da270 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f55f9ffb640 1 -- 172.21.15.88:0/3827683933 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56140053c0 con 0x55ecf07da270 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f55f9ffb640 1 -- 172.21.15.88:0/3827683933 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 13) ==== 4752+0+0 (secure 0 0 0) 0x7f561402e020 con 0x55ecf07da270 2024-09-21T02:04:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 10 client.4734 fetch_fsmap finished waiting for FSMap version 13 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 10 client.4734 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 20 client.4734 populate_metadata read hostname 'smithi088' 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0798e40 0x55ecf0be0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 4 client.4734 mds_command: new command op to 4398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 --> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecf0af6770 con 0x55ecf0798e40 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.066+0000 7f561affd640 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0798e40 0x55ecf0be0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.067+0000 7f55f9ffb640 10 client.4734 ms_handle_connect on v2:172.21.15.195:6836/778249012 2024-09-21T02:04:18.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:18.067+0000 7f561affd640 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0798e40 0x55ecf0be0550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6302ce7f8e7c9d29 server_cookie=7206c11feac8f075 in_seq=0 out_seq=0 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f55f9ffb640 1 -- 172.21.15.88:0/3827683933 <== mds.2 v2:172.21.15.195:6836/778249012 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ecf0af6770 con 0x55ecf0798e40 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f55f9ffb640 10 client.4734 handle_command_reply: tid=0 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 client.4734 shutdown 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 msgr2=0x55ecf0bc0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bc0500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5614009010 tx=0x7f561400dd00 comp rx=0 tx=0).stop 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 shutdown_connections 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0798e40 0x55ecf0be0550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bc0500 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bb2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 --2- 172.21.15.88:0/3827683933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0bb90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 >> 172.21.15.88:0/3827683933 conn(0x55ecf07ce3c0 msgr2=0x55ecf0b7c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f55f9ffb640 0 client.4734 ms_handle_reset on v2:172.21.15.195:6836/778249012 2024-09-21T02:04:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.149+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 shutdown_connections 2024-09-21T02:04:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.150+0000 7f56286c9740 1 -- 172.21.15.88:0/3827683933 wait complete. 2024-09-21T02:04:21.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.150+0000 7f56286c9740 20 client.4734 trim_cache size 0 max 16384 2024-09-21T02:04:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.170+0000 7f56286c9740 1 Processor -- start 2024-09-21T02:04:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.170+0000 7f56286c9740 1 -- start start 2024-09-21T02:04:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b80ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f56286c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f56286c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f561c149690 con 0x55ecf07b9590 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecf0bcbef0 con 0x55ecf07da270 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ecf0bb98c0 con 0x55ecf07b13b0 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f561a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.171+0000 7f561a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49548/0 (socket says 172.21.15.88:49548) 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561a7fc640 1 -- 172.21.15.88:0/356438750 learned_addr learned my addr 172.21.15.88:0/356438750 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561affd640 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b80ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561b7fe640 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561a7fc640 1 -- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 msgr2=0x55ecf0b80ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561a7fc640 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b80ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561a7fc640 1 -- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 msgr2=0x55ecf0bac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561a7fc640 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561a7fc640 1 -- 172.21.15.88:0/356438750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf0b7f7b0 con 0x55ecf07b9590 2024-09-21T02:04:21.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f561a7fc640 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b811e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f56100b9890 tx=0x7f56100b97c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf4f66258e557fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f55fbfff640 1 -- 172.21.15.88:0/356438750 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610048040 con 0x55ecf07b9590 2024-09-21T02:04:21.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f55fbfff640 1 -- 172.21.15.88:0/356438750 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5610082c10 con 0x55ecf07b9590 2024-09-21T02:04:21.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.172+0000 7f55fbfff640 1 -- 172.21.15.88:0/356438750 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610005310 con 0x55ecf07b9590 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 -- 172.21.15.88:0/356438750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 msgr2=0x55ecf0b811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b811e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f56100b9890 tx=0x7f56100b97c0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 -- 172.21.15.88:0/356438750 shutdown_connections 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07da270 0x55ecf0bac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07b9590 0x55ecf0b811e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 --2- 172.21.15.88:0/356438750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b13b0 0x55ecf0b80ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 -- 172.21.15.88:0/356438750 >> 172.21.15.88:0/356438750 conn(0x55ecf07ce3c0 msgr2=0x55ecf0bd4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 -- 172.21.15.88:0/356438750 shutdown_connections 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.173+0000 7f56286c9740 1 -- 172.21.15.88:0/356438750 wait complete. 2024-09-21T02:04:21.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.174+0000 7f56286c9740 1 -- start start 2024-09-21T02:04:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.174+0000 7f56286c9740 1 Processor -- start 2024-09-21T02:04:21.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0c3bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f56286c9740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f56286c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f56286c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ecf0bbb6c0 con 0x55ecf07da270 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecf0bbb090 con 0x55ecf07b13b0 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f56286c9740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x55ecf0b7b440 con 0x55ecf07b9590 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f56286c9740 10 client.0 _ll_register_callbacks cb 0x7f5614016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bba670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49562/0 (socket says 172.21.15.88:49562) 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 -- 172.21.15.88:0/2057440507 learned_addr learned my addr 172.21.15.88:0/2057440507 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561affd640 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0c3bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561a7fc640 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 -- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 msgr2=0x55ecf0bba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 -- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 msgr2=0x55ecf0c3bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0c3bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 -- 172.21.15.88:0/2057440507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf0bbff80 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f561b7fe640 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bba670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ecf0b80af0 tx=0x7f5614003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6ff58779020fef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f55f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.176+0000 7f55f9ffb640 1 -- 172.21.15.88:0/2057440507 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614018040 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f55f9ffb640 1 -- 172.21.15.88:0/2057440507 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f561400da40 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5610082c10 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f55f9ffb640 1 -- 172.21.15.88:0/2057440507 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f561403c050 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 10 client.4740 fetch_fsmap learned FSMap version 13 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x55ecf0bb0330 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f55f9ffb640 1 -- 172.21.15.88:0/2057440507 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5610082c10 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f55f9ffb640 1 -- 172.21.15.88:0/2057440507 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 13) ==== 4752+0+0 (secure 0 0 0) 0x7f561403f020 con 0x55ecf07da270 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 10 client.4740 fetch_fsmap finished waiting for FSMap version 13 2024-09-21T02:04:21.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 10 client.4740 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 20 client.4740 populate_metadata read hostname 'smithi088' 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0bcb1b0 0x55ecf0bafa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 4 client.4740 mds_command: new command op to 4398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 --> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] -- command(tid 0: {"prefix": "status"}) -- 0x55ecf0af2220 con 0x55ecf0bcb1b0 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.177+0000 7f561affd640 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0bcb1b0 0x55ecf0bafa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f55f9ffb640 10 client.4740 ms_handle_connect on v2:172.21.15.195:6836/778249012 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f561affd640 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0bcb1b0 0x55ecf0bafa80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22c5f418d921bf8c server_cookie=bb84d964afa853bc in_seq=0 out_seq=0 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f55f9ffb640 1 -- 172.21.15.88:0/2057440507 <== mds.2 v2:172.21.15.195:6836/778249012 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ecf0af2220 con 0x55ecf0bcb1b0 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f55f9ffb640 10 client.4740 handle_command_reply: tid=0 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 client.4740 shutdown 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 msgr2=0x55ecf0bba670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bba670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ecf0b80af0 tx=0x7f5614003f50 comp rx=0 tx=0).stop 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 shutdown_connections 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:6836/778249012,v1:172.21.15.195:6837/778249012] conn(0x55ecf0bcb1b0 0x55ecf0bafa80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ecf07da270 0x55ecf0bba670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecf07b9590 0x55ecf0bba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f55f9ffb640 0 client.4740 ms_handle_reset on v2:172.21.15.195:6836/778249012 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 --2- 172.21.15.88:0/2057440507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecf07b13b0 0x55ecf0c3bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 >> 172.21.15.88:0/2057440507 conn(0x55ecf07ce3c0 msgr2=0x55ecf0bd4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 shutdown_connections 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.178+0000 7f56286c9740 1 -- 172.21.15.88:0/2057440507 wait complete. 2024-09-21T02:04:21.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f56286c9740 20 client.4740 trim_cache size 0 max 16384 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 2, 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4398, 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 11.049649605000001, 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 13, 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 21, 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 21, 2024-09-21T02:04:21.185 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 29.125776158000001 2024-09-21T02:04:21.186 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:21.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5608078bc0 msgr2=0x7f560807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5608078bc0 0x7f560807b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f560c005fd0 tx=0x7f560c005f40 comp rx=0 tx=0).stop 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 msgr2=0x7f561c14f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c14f4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5610000c00 tx=0x7f56100038a0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 shutdown_connections 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5608078bc0 0x7f560807b0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f561c15b980 0x7f561c152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f561c0cb990 0x7f561c14f4a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 --2- 172.21.15.88:0/2159322585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f561c0b51f0 0x7f561c14ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 >> 172.21.15.88:0/2159322585 conn(0x7f561c0cda90 msgr2=0x7f561c06cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 shutdown_connections 2024-09-21T02:04:21.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.179+0000 7f562175f640 1 -- 172.21.15.88:0/2159322585 wait complete. 2024-09-21T02:04:21.194 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T02:04:21.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.312+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/624426821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c004e00 msgr2=0x7fd99c0052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.312+0000 7fd9b8a0a640 1 --2- 172.21.15.88:0/624426821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c004e00 0x7fd99c0052d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac005fd0 tx=0x7fd99c005830 comp rx=0 tx=0).stop 2024-09-21T02:04:21.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.312+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/624426821 reap_dead start 2024-09-21T02:04:21.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.313+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/624426821 shutdown_connections 2024-09-21T02:04:21.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.313+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/624426821 >> 172.21.15.88:0/624426821 conn(0x7fd9b4070960 msgr2=0x7fd9b406cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.313+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/624426821 shutdown_connections 2024-09-21T02:04:21.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.313+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/624426821 wait complete. 2024-09-21T02:04:21.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.314+0000 7fd9b8a0a640 1 Processor -- start 2024-09-21T02:04:21.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.315+0000 7fd9b8a0a640 1 -- start start 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b8a0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd99c004e00 0x7fd9b414b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b8a0a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4154e10 0x7fd9b415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b8a0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9b415c080 0x7fd9b4155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b8a0a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4159bf0 con 0x7fd9b4154e10 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b8a0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b415a240 con 0x7fd9b415c080 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b8a0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b4159ed0 con 0x7fd99c004e00 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4154e10 0x7fd9b415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4154e10 0x7fd9b415bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49584/0 (socket says 172.21.15.88:49584) 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/3105381945 learned_addr learned my addr 172.21.15.88:0/3105381945 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b259c640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd99c004e00 0x7fd9b414b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9b415c080 0x7fd9b4155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd99c004e00 msgr2=0x7fd9b414b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd99c004e00 0x7fd9b414b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9b415c080 msgr2=0x7fd9b4155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9b415c080 0x7fd9b4155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/3105381945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b40c8560 con 0x7fd9b4154e10 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9b415c080 0x7fd9b4155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4154e10 0x7fd9b415bb40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006630 tx=0x7fd9a80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9a2ffd640 1 -- 172.21.15.88:0/3105381945 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8015070 con 0x7fd9b4154e10 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9a2ffd640 1 -- 172.21.15.88:0/3105381945 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9a8011900 con 0x7fd9b4154e10 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b40c7880 con 0x7fd9b4154e10 2024-09-21T02:04:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.316+0000 7fd9a2ffd640 1 -- 172.21.15.88:0/3105381945 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8011d50 con 0x7fd9b4154e10 2024-09-21T02:04:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.317+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41531d0 con 0x7fd9b4154e10 2024-09-21T02:04:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.318+0000 7fd9a2ffd640 1 -- 172.21.15.88:0/3105381945 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9a8018070 con 0x7fd9b4154e10 2024-09-21T02:04:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.319+0000 7fd9a2ffd640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd994078bc0 0x7fd99407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.319+0000 7fd9a2ffd640 1 -- 172.21.15.88:0/3105381945 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd9a809ba70 con 0x7fd9b4154e10 2024-09-21T02:04:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.319+0000 7fd9b259c640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd994078bc0 0x7fd99407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.320+0000 7fd9b259c640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd994078bc0 0x7fd99407b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4006580 tx=0x7fd9a4006b10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.320+0000 7fd9bfa53740 1 Processor -- start 2024-09-21T02:04:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.320+0000 7fd9bfa53740 1 -- start start 2024-09-21T02:04:21.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e3e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5df4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5df9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5572e5d6c820 con 0x5572e5a52270 2024-09-21T02:04:21.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5572e5d5bc50 con 0x5572e5a31590 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5572e5d5fa60 con 0x5572e5a293b0 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e3e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e3e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49600/0 (socket says 172.21.15.88:49600) 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 -- 172.21.15.88:0/4104163208 learned_addr learned my addr 172.21.15.88:0/4104163208 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5df4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5df9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 -- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 msgr2=0x5572e5df4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5df4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 -- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 msgr2=0x5572e5df9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5df9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b259c640 1 -- 172.21.15.88:0/4104163208 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e5abd850 con 0x5572e5a52270 2024-09-21T02:04:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.321+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5df9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.322+0000 7fd9b259c640 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e3e1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4001930 tx=0x7fd9a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9207ece64486cf77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.322+0000 7fd993fff640 1 -- 172.21.15.88:0/4104163208 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4012a80 con 0x5572e5a52270 2024-09-21T02:04:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.322+0000 7fd993fff640 1 -- 172.21.15.88:0/4104163208 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9a4024020 con 0x5572e5a52270 2024-09-21T02:04:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.322+0000 7fd993fff640 1 -- 172.21.15.88:0/4104163208 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4012e60 con 0x5572e5a52270 2024-09-21T02:04:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 -- 172.21.15.88:0/4104163208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 msgr2=0x5572e5e3e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e3e1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4001930 tx=0x7fd9a4015000 comp rx=0 tx=0).stop 2024-09-21T02:04:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 -- 172.21.15.88:0/4104163208 shutdown_connections 2024-09-21T02:04:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5df9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5df4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/4104163208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e3e1e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 -- 172.21.15.88:0/4104163208 >> 172.21.15.88:0/4104163208 conn(0x5572e5a463c0 msgr2=0x5572e5e3d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 -- 172.21.15.88:0/4104163208 shutdown_connections 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.323+0000 7fd9bfa53740 1 -- 172.21.15.88:0/4104163208 wait complete. 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.324+0000 7fd9bfa53740 1 -- start start 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.324+0000 7fd9bfa53740 1 Processor -- start 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a31590 0x5572e5e2abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a52270 0x5572e5e31620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5572e5acc620 con 0x5572e5a31590 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5572e5d015f0 con 0x5572e5a52270 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5572e5e35ea0 con 0x5572e5a293b0 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9bfa53740 10 client.0 _ll_register_callbacks cb 0x5572e5d050a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a31590 0x5572e5e2abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a31590 0x5572e5e2abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49614/0 (socket says 172.21.15.88:49614) 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/213502578 learned_addr learned my addr 172.21.15.88:0/213502578 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b259c640 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a52270 0x5572e5e31620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 msgr2=0x5572e5e49990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e49990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a52270 msgr2=0x5572e5e31620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a52270 0x5572e5e31620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.325+0000 7fd9b1d9b640 1 -- 172.21.15.88:0/213502578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e5b346b0 con 0x5572e5a31590 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a31590 0x5572e5e2abe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8060ae0 tx=0x7fd9a8063e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e9e101a5c237c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9917fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9917fa640 1 -- 172.21.15.88:0/213502578 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a801f040 con 0x5572e5a31590 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9917fa640 1 -- 172.21.15.88:0/213502578 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd9a8064af0 con 0x5572e5a31590 2024-09-21T02:04:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a4024020 con 0x5572e5a31590 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9917fa640 1 -- 172.21.15.88:0/213502578 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7fd9a8063380 con 0x5572e5a31590 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 10 client.4752 fetch_fsmap learned FSMap version 13 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x5572e5e49470 con 0x5572e5a31590 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9917fa640 1 -- 172.21.15.88:0/213502578 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9a4024020 con 0x5572e5a31590 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9917fa640 1 -- 172.21.15.88:0/213502578 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 13) ==== 4752+0+0 (secure 0 0 0) 0x7fd9a807abd0 con 0x5572e5a31590 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 10 client.4752 fetch_fsmap finished waiting for FSMap version 13 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 10 client.4752 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 20 client.4752 populate_metadata read hostname 'smithi088' 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5a10e40 0x5572e5e57ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 4 client.4752 mds_command: new command op to 4399 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 --> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572e5d6e770 con 0x5572e5a10e40 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.326+0000 7fd9b259c640 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5a10e40 0x5572e5e57ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.327+0000 7fd9917fa640 10 client.4752 ms_handle_connect on v2:172.21.15.88:6836/3330183943 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.327+0000 7fd9b259c640 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5a10e40 0x5572e5e57ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b65e897917d2d3c server_cookie=7544e96ca95fbf65 in_seq=0 out_seq=0 2024-09-21T02:04:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.327+0000 7fd9917fa640 1 -- 172.21.15.88:0/213502578 <== mds.1 v2:172.21.15.88:6836/3330183943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572e5d6e770 con 0x5572e5a10e40 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.327+0000 7fd9917fa640 10 client.4752 handle_command_reply: tid=0 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.327+0000 7fd9bfa53740 1 client.4752 shutdown 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a31590 msgr2=0x5572e5e2abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a31590 0x5572e5e2abe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8060ae0 tx=0x7fd9a8063e10 comp rx=0 tx=0).stop 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 shutdown_connections 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a52270 0x5572e5e31620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a31590 0x5572e5e2abe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5a10e40 0x5572e5e57ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/213502578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e49990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 >> 172.21.15.88:0/213502578 conn(0x5572e5a463c0 msgr2=0x5572e5df4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9917fa640 0 client.4752 ms_handle_reset on v2:172.21.15.88:6836/3330183943 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 shutdown_connections 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 1 -- 172.21.15.88:0/213502578 wait complete. 2024-09-21T02:04:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.328+0000 7fd9bfa53740 20 client.4752 trim_cache size 0 max 16384 2024-09-21T02:04:21.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.346+0000 7fd9bfa53740 1 Processor -- start 2024-09-21T02:04:21.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.346+0000 7fd9bfa53740 1 -- start start 2024-09-21T02:04:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5e397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e39cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41fb370 con 0x5572e5a52270 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5572e5e30c40 con 0x5572e5a31590 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5572e5e30690 con 0x5572e5a293b0 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e39cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e39cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49616/0 (socket says 172.21.15.88:49616) 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.347+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/3662594098 learned_addr learned my addr 172.21.15.88:0/3662594098 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b259c640 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5e397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 msgr2=0x5572e5e33e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e33e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 msgr2=0x5572e5e397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5e397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/3662594098 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e5df8500 con 0x5572e5a52270 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b259c640 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e33e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e39cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009f90 tx=0x7fd9ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b79ef1c76a8361 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9937fe640 1 -- 172.21.15.88:0/3662594098 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac046070 con 0x5572e5a52270 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9937fe640 1 -- 172.21.15.88:0/3662594098 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ac003c60 con 0x5572e5a52270 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.348+0000 7fd9937fe640 1 -- 172.21.15.88:0/3662594098 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac038040 con 0x5572e5a52270 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 -- 172.21.15.88:0/3662594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 msgr2=0x5572e5e39cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e39cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009f90 tx=0x7fd9ac004ab0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 -- 172.21.15.88:0/3662594098 shutdown_connections 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e39cf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a31590 0x5572e5e397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/3662594098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a293b0 0x5572e5e33e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 -- 172.21.15.88:0/3662594098 >> 172.21.15.88:0/3662594098 conn(0x5572e5a463c0 msgr2=0x5572e5e2f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 -- 172.21.15.88:0/3662594098 shutdown_connections 2024-09-21T02:04:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.349+0000 7fd9bfa53740 1 -- 172.21.15.88:0/3662594098 wait complete. 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.350+0000 7fd9bfa53740 1 -- start start 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.350+0000 7fd9bfa53740 1 Processor -- start 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a293b0 0x5572e5df39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a31590 0x5572e5e277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9bfa53740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e36cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5572e5e32f70 con 0x5572e5a52270 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5572e5e32920 con 0x5572e5a293b0 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9bfa53740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x5572e5e4cae0 con 0x5572e5a31590 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9bfa53740 10 client.0 _ll_register_callbacks cb 0x7fd9a805f730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e36cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e36cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49632/0 (socket says 172.21.15.88:49632) 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/953642788 learned_addr learned my addr 172.21.15.88:0/953642788 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9b259c640 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a293b0 0x5572e5df39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.351+0000 7fd9b1d9b640 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a31590 0x5572e5e277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a31590 msgr2=0x5572e5e277e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a31590 0x5572e5e277e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a293b0 msgr2=0x5572e5df39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a293b0 0x5572e5df39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9b2d9d640 1 -- 172.21.15.88:0/953642788 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e5e33460 con 0x5572e5a52270 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9b2d9d640 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e36cf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009e20 tx=0x7fd9ac004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754a87aa0af8f5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9917fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9917fa640 1 -- 172.21.15.88:0/953642788 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac046050 con 0x5572e5a52270 2024-09-21T02:04:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9917fa640 1 -- 172.21.15.88:0/953642788 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd9ac004580 con 0x5572e5a52270 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac003c60 con 0x5572e5a52270 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9917fa640 1 -- 172.21.15.88:0/953642788 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac038040 con 0x5572e5a52270 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 10 client.4758 fetch_fsmap learned FSMap version 13 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x5572e5e36620 con 0x5572e5a52270 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9917fa640 1 -- 172.21.15.88:0/953642788 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ac003c60 con 0x5572e5a52270 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9917fa640 1 -- 172.21.15.88:0/953642788 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 13) ==== 4752+0+0 (secure 0 0 0) 0x7fd9ac03d440 con 0x5572e5a52270 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 10 client.4758 fetch_fsmap finished waiting for FSMap version 13 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 10 client.4758 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 20 client.4758 populate_metadata read hostname 'smithi088' 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5e460a0 0x5572e5e464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 4 client.4758 mds_command: new command op to 4399 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.352+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 --> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] -- command(tid 0: {"prefix": "status"}) -- 0x5572e5d6a220 con 0x5572e5e460a0 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9b259c640 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5e460a0 0x5572e5e464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9917fa640 10 client.4758 ms_handle_connect on v2:172.21.15.88:6836/3330183943 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9b259c640 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5e460a0 0x5572e5e464b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=395a545bcd4834b9 server_cookie=8b35690792318d01 in_seq=0 out_seq=0 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9917fa640 1 -- 172.21.15.88:0/953642788 <== mds.1 v2:172.21.15.88:6836/3330183943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5572e5d6a220 con 0x5572e5e460a0 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9917fa640 10 client.4758 handle_command_reply: tid=0 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 client.4758 shutdown 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 msgr2=0x5572e5e36cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e36cf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009e20 tx=0x7fd9ac004410 comp rx=0 tx=0).stop 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 shutdown_connections 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572e5a52270 0x5572e5e36cf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.88:6836/3330183943,v1:172.21.15.88:6837/3330183943] conn(0x5572e5e460a0 0x5572e5e464b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5572e5a31590 0x5572e5e277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 --2- 172.21.15.88:0/953642788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5572e5a293b0 0x5572e5df39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9917fa640 0 client.4758 ms_handle_reset on v2:172.21.15.88:6836/3330183943 2024-09-21T02:04:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 >> 172.21.15.88:0/953642788 conn(0x5572e5a463c0 msgr2=0x5572e5e31340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.353+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 shutdown_connections 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9bfa53740 1 -- 172.21.15.88:0/953642788 wait complete. 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9bfa53740 20 client.4758 trim_cache size 0 max 16384 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 1, 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4399, 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T02:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T02:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 18.256967634999999, 2024-09-21T02:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 13, 2024-09-21T02:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 21, 2024-09-21T02:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 21, 2024-09-21T02:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 29.321134086000001 2024-09-21T02:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd994078bc0 msgr2=0x7fd99407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd994078bc0 0x7fd99407b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4006580 tx=0x7fd9a4006b10 comp rx=0 tx=0).stop 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4154e10 msgr2=0x7fd9b415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4154e10 0x7fd9b415bb40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006630 tx=0x7fd9a80039c0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 shutdown_connections 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd994078bc0 0x7fd99407b0d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9b415c080 0x7fd9b4155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b4154e10 0x7fd9b415bb40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 --2- 172.21.15.88:0/3105381945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd99c004e00 0x7fd9b414b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 >> 172.21.15.88:0/3105381945 conn(0x7fd9b4070960 msgr2=0x7fd9b4157c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 shutdown_connections 2024-09-21T02:04:21.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.354+0000 7fd9b8a0a640 1 -- 172.21.15.88:0/3105381945 wait complete. 2024-09-21T02:04:21.366 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T02:04:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.478+0000 7fa3f6790640 1 -- 172.21.15.88:0/2306646708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 msgr2=0x7fa3d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.478+0000 7fa3f6790640 1 --2- 172.21.15.88:0/2306646708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 0x7fa3d8004800 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4001800 tx=0x7fa3d8005830 comp rx=0 tx=0).stop 2024-09-21T02:04:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.478+0000 7fa3f4d2d640 1 -- 172.21.15.88:0/2306646708 reap_dead start 2024-09-21T02:04:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.478+0000 7fa3f6790640 1 -- 172.21.15.88:0/2306646708 shutdown_connections 2024-09-21T02:04:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.478+0000 7fa3f6790640 1 -- 172.21.15.88:0/2306646708 >> 172.21.15.88:0/2306646708 conn(0x7fa3f006d8f0 msgr2=0x7fa3f00ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.478+0000 7fa3f6790640 1 -- 172.21.15.88:0/2306646708 shutdown_connections 2024-09-21T02:04:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.478+0000 7fa3f6790640 1 -- 172.21.15.88:0/2306646708 wait complete. 2024-09-21T02:04:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.480+0000 7fa3f6790640 1 Processor -- start 2024-09-21T02:04:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.480+0000 7fa3f6790640 1 -- start start 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 0x7fa3f014cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3f0157fe0 0x7fa3f01497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f014db70 0x7fa3f014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0075940 con 0x7fa3d80041c0 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f00c9710 con 0x7fa3f014db70 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f00c1ec0 con 0x7fa3f0157fe0 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 0x7fa3f014cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 0x7fa3f014cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49640/0 (socket says 172.21.15.88:49640) 2024-09-21T02:04:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 -- 172.21.15.88:0/3697186804 learned_addr learned my addr 172.21.15.88:0/3697186804 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f4d2d640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f014db70 0x7fa3f014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3f0157fe0 0x7fa3f01497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 -- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3f0157fe0 msgr2=0x7fa3f01497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3f0157fe0 0x7fa3f01497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 -- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f014db70 msgr2=0x7fa3f014df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f014db70 0x7fa3f014df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 -- 172.21.15.88:0/3697186804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f014d2e0 con 0x7fa3d80041c0 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f4d2d640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f014db70 0x7fa3f014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3effff640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 0x7fa3f014cd80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0011b10 tx=0x7fa3e0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3ecff9640 1 -- 172.21.15.88:0/3697186804 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001e070 con 0x7fa3d80041c0 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3ecff9640 1 -- 172.21.15.88:0/3697186804 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e0004050 con 0x7fa3d80041c0 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01f3a70 con 0x7fa3d80041c0 2024-09-21T02:04:21.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3ecff9640 1 -- 172.21.15.88:0/3697186804 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000fea0 con 0x7fa3d80041c0 2024-09-21T02:04:21.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.482+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01f3d30 con 0x7fa3d80041c0 2024-09-21T02:04:21.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.484+0000 7fa3ecff9640 1 -- 172.21.15.88:0/3697186804 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3e0021070 con 0x7fa3d80041c0 2024-09-21T02:04:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.485+0000 7fa3ecff9640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3dc078b90 0x7fa3dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.485+0000 7fa3ecff9640 1 -- 172.21.15.88:0/3697186804 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa3e009b7d0 con 0x7fa3d80041c0 2024-09-21T02:04:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.485+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3dc078b90 0x7fa3dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.485+0000 7fa3fd7e0740 1 Processor -- start 2024-09-21T02:04:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.485+0000 7fa3fd7e0740 1 -- start start 2024-09-21T02:04:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.486+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3dc078b90 0x7fa3dc07b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4032000 tx=0x7fa3e4001360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9b00d270 0x555d9b3f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afe43b0 0x555d9b3af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9afec590 0x555d9b3b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555d9b327820 con 0x555d9afe43b0 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d9b316c50 con 0x555d9afec590 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555d9b31aa60 con 0x555d9b00d270 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afe43b0 0x555d9b3af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afe43b0 0x555d9b3af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49654/0 (socket says 172.21.15.88:49654) 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/43441251 learned_addr learned my addr 172.21.15.88:0/43441251 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3f4d2d640 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9afec590 0x555d9b3b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3effff640 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9b00d270 0x555d9b3f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9b00d270 msgr2=0x555d9b3f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9b00d270 0x555d9b3f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9afec590 msgr2=0x555d9b3b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9afec590 0x555d9b3b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/43441251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9b078850 con 0x555d9afe43b0 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afe43b0 0x555d9b3af940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40018f0 tx=0x7fa3e403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8901423941d3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3d5ffb640 1 -- 172.21.15.88:0/43441251 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4033680 con 0x555d9afe43b0 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3d5ffb640 1 -- 172.21.15.88:0/43441251 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e4033910 con 0x555d9afe43b0 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.487+0000 7fa3d5ffb640 1 -- 172.21.15.88:0/43441251 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4033cc0 con 0x555d9afe43b0 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/43441251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afe43b0 msgr2=0x555d9b3af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afe43b0 0x555d9b3af940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40018f0 tx=0x7fa3e403d000 comp rx=0 tx=0).stop 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/43441251 shutdown_connections 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9afec590 0x555d9b3b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afe43b0 0x555d9b3af940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/43441251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9b00d270 0x555d9b3f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/43441251 >> 172.21.15.88:0/43441251 conn(0x555d9b0013c0 msgr2=0x555d9b3f8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/43441251 shutdown_connections 2024-09-21T02:04:21.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.488+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/43441251 wait complete. 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.489+0000 7fa3fd7e0740 1 -- start start 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.489+0000 7fa3fd7e0740 1 Processor -- start 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.490+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555d9b087620 con 0x555d9afec590 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d9b2bc5f0 con 0x555d9b00d270 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555d9b3ec930 con 0x555d9afe43b0 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 10 client.0 _ll_register_callbacks cb 0x555d9b2c00a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49670/0 (socket says 172.21.15.88:49670) 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1985946442 learned_addr learned my addr 172.21.15.88:0/1985946442 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3f4d2d640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3effff640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 msgr2=0x555d9b3ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3ebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 msgr2=0x555d9b3e58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3e58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1985946442 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9b0ef6b0 con 0x555d9afec590 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3f4d2d640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3ae5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4040000 tx=0x7fa3e4006290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee63ddee037b2de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T02:04:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ceffd640 1 -- 172.21.15.88:0/1985946442 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4033cc0 con 0x555d9afec590 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ceffd640 1 -- 172.21.15.88:0/1985946442 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa3e403f460 con 0x555d9afec590 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e4033910 con 0x555d9afec590 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3ceffd640 1 -- 172.21.15.88:0/1985946442 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7fa3e405f050 con 0x555d9afec590 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 10 client.4770 fetch_fsmap learned FSMap version 13 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.491+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x555d9b3af370 con 0x555d9afec590 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3ceffd640 1 -- 172.21.15.88:0/1985946442 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e4033910 con 0x555d9afec590 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3ceffd640 1 -- 172.21.15.88:0/1985946442 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 13) ==== 4752+0+0 (secure 0 0 0) 0x7fa3e4062020 con 0x555d9afec590 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3fd7e0740 10 client.4770 fetch_fsmap finished waiting for FSMap version 13 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3fd7e0740 10 client.4770 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3fd7e0740 20 client.4770 populate_metadata read hostname 'smithi088' 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9afcbe40 0x555d9b4135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3fd7e0740 4 client.4770 mds_command: new command op to 4401 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:04:21.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 --> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d9b329770 con 0x555d9afcbe40 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3effff640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9afcbe40 0x555d9b4135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3effff640 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9afcbe40 0x555d9b4135a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43c7e558e250b48 server_cookie=5ae0c550c9f2e03b in_seq=0 out_seq=0 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.492+0000 7fa3ceffd640 10 client.4770 ms_handle_connect on v2:172.21.15.195:6838/3087611678 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.493+0000 7fa3ceffd640 1 -- 172.21.15.88:0/1985946442 <== mds.0 v2:172.21.15.195:6838/3087611678 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d9b329770 con 0x555d9afcbe40 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.493+0000 7fa3ceffd640 10 client.4770 handle_command_reply: tid=0 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.493+0000 7fa3fd7e0740 1 client.4770 shutdown 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 msgr2=0x555d9b3ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3ae5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4040000 tx=0x7fa3e4006290 comp rx=0 tx=0).stop 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 shutdown_connections 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9afcbe40 0x555d9b4135a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3ceffd640 0 client.4770 ms_handle_reset on v2:172.21.15.195:6838/3087611678 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3ae5f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1985946442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 >> 172.21.15.88:0/1985946442 conn(0x555d9b0013c0 msgr2=0x555d9b3b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 shutdown_connections 2024-09-21T02:04:21.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1985946442 wait complete. 2024-09-21T02:04:21.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.494+0000 7fa3fd7e0740 20 client.4770 trim_cache size 0 max 16384 2024-09-21T02:04:21.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.512+0000 7fa3fd7e0740 1 Processor -- start 2024-09-21T02:04:21.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.512+0000 7fa3fd7e0740 1 -- start start 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3fed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f006f390 con 0x555d9afec590 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d9b087620 con 0x555d9b00d270 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555d9b3ec6c0 con 0x555d9afe43b0 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3fe810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49678/0 (socket says 172.21.15.88:49678) 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1413643873 learned_addr learned my addr 172.21.15.88:0/1413643873 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3f4d2d640 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3effff640 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 msgr2=0x555d9b3af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 msgr2=0x555d9b3fed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3fed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.513+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/1413643873 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9b3b0290 con 0x555d9afec590 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3fe810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4003e50 tx=0x7fa3e4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7215f779b7feb489 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3d57fa640 1 -- 172.21.15.88:0/1413643873 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4033b10 con 0x555d9afec590 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3d57fa640 1 -- 172.21.15.88:0/1413643873 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e403f840 con 0x555d9afec590 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3d57fa640 1 -- 172.21.15.88:0/1413643873 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e403fc90 con 0x555d9afec590 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1413643873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 msgr2=0x555d9b3fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3fe810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4003e50 tx=0x7fa3e4004f90 comp rx=0 tx=0).stop 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1413643873 shutdown_connections 2024-09-21T02:04:21.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b3fe810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/1413643873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1413643873 >> 172.21.15.88:0/1413643873 conn(0x555d9b0013c0 msgr2=0x555d9b3e5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.514+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1413643873 shutdown_connections 2024-09-21T02:04:21.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.515+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/1413643873 wait complete. 2024-09-21T02:04:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.515+0000 7fa3fd7e0740 1 -- start start 2024-09-21T02:04:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.516+0000 7fa3fd7e0740 1 Processor -- start 2024-09-21T02:04:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b46f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555d9b3e5de0 con 0x555d9afec590 2024-09-21T02:04:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555d9b3ee8f0 con 0x555d9b00d270 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x555d9b407f00 con 0x555d9afe43b0 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 10 client.0 _ll_register_callbacks cb 0x7fa3e8001250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b46f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b46f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49682/0 (socket says 172.21.15.88:49682) 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/3422160579 learned_addr learned my addr 172.21.15.88:0/3422160579 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3effff640 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3f4d2d640 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 msgr2=0x555d9b3f0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3f0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 msgr2=0x555d9b3af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 -- 172.21.15.88:0/3422160579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9b3f0fd0 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ef7fe640 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b46f250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40064c0 tx=0x7fa3e403fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cf486c07fb95f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ceffd640 1 -- 172.21.15.88:0/3422160579 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e404d650 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3ceffd640 1 -- 172.21.15.88:0/3422160579 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa3e404d7f0 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.517+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e403f840 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3ceffd640 1 -- 172.21.15.88:0/3422160579 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7fa3e4070050 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 10 client.4776 fetch_fsmap learned FSMap version 13 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x555d9b3f0fd0 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3ceffd640 1 -- 172.21.15.88:0/3422160579 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e403f840 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3ceffd640 1 -- 172.21.15.88:0/3422160579 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 13) ==== 4752+0+0 (secure 0 0 0) 0x7fa3e4073020 con 0x555d9afec590 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 10 client.4776 fetch_fsmap finished waiting for FSMap version 13 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 10 client.4776 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T02:04:21.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 20 client.4776 populate_metadata read hostname 'smithi088' 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9b4754c0 0x555d9b4758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 4 client.4776 mds_command: new command op to 4401 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 --> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] -- command(tid 0: {"prefix": "status"}) -- 0x555d9b3ef1f0 con 0x555d9b4754c0 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3effff640 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9b4754c0 0x555d9b4758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3effff640 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9b4754c0 0x555d9b4758d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6acf710b099612 server_cookie=c01b0365da8f4161 in_seq=0 out_seq=0 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.518+0000 7fa3ceffd640 10 client.4776 ms_handle_connect on v2:172.21.15.195:6838/3087611678 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3ceffd640 1 -- 172.21.15.88:0/3422160579 <== mds.0 v2:172.21.15.195:6838/3087611678 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x555d9b3ef1f0 con 0x555d9b4754c0 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3ceffd640 10 client.4776 handle_command_reply: tid=0 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 client.4776 shutdown 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 msgr2=0x555d9b46f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b46f250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40064c0 tx=0x7fa3e403fc80 comp rx=0 tx=0).stop 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 shutdown_connections 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:6838/3087611678,v1:172.21.15.195:6839/3087611678] conn(0x555d9b4754c0 0x555d9b4758d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d9b00d270 0x555d9b3af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555d9afec590 0x555d9b46f250 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 --2- 172.21.15.88:0/3422160579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d9afe43b0 0x555d9b3f0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3ceffd640 0 client.4776 ms_handle_reset on v2:172.21.15.195:6838/3087611678 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 >> 172.21.15.88:0/3422160579 conn(0x555d9b0013c0 msgr2=0x555d9b3fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 shutdown_connections 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 1 -- 172.21.15.88:0/3422160579 wait complete. 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.519+0000 7fa3fd7e0740 20 client.4776 trim_cache size 0 max 16384 2024-09-21T02:04:21.526 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4401, 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 23.461123917999998, 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 13, 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 21, 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 21, 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 29.464293762 2024-09-21T02:04:21.527 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3dc078b90 msgr2=0x7fa3dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3dc078b90 0x7fa3dc07b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4032000 tx=0x7fa3e4001360 comp rx=0 tx=0).stop 2024-09-21T02:04:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 msgr2=0x7fa3f014cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 0x7fa3f014cd80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0011b10 tx=0x7fa3e0009e90 comp rx=0 tx=0).stop 2024-09-21T02:04:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 shutdown_connections 2024-09-21T02:04:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3dc078b90 0x7fa3dc07b0a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3f014db70 0x7fa3f014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3f0157fe0 0x7fa3f01497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 --2- 172.21.15.88:0/3697186804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3d80041c0 0x7fa3f014cd80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 >> 172.21.15.88:0/3697186804 conn(0x7fa3f006d8f0 msgr2=0x7fa3f014ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 shutdown_connections 2024-09-21T02:04:21.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.520+0000 7fa3f6790640 1 -- 172.21.15.88:0/3697186804 wait complete. 2024-09-21T02:04:21.532 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T02:04:21.557 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T02:04:21.557 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T02:04:21.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:21.557 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T02:04:21.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:21.574 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T02:04:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-09-21T02:04:21.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:04:21.642 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T02:04:21.642 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:04:21.642 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:21.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:04:21.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.793+0000 7f7694030640 1 -- 172.21.15.88:0/142878691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 msgr2=0x7f7678004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.793+0000 7f7694030640 1 --2- 172.21.15.88:0/142878691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 0x7f7678004c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f767c009d20 tx=0x7f7678005830 comp rx=0 tx=0).stop 2024-09-21T02:04:21.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.793+0000 7f76925cd640 1 -- 172.21.15.88:0/142878691 reap_dead start 2024-09-21T02:04:21.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.793+0000 7f7694030640 1 -- 172.21.15.88:0/142878691 shutdown_connections 2024-09-21T02:04:21.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.793+0000 7f7694030640 1 -- 172.21.15.88:0/142878691 >> 172.21.15.88:0/142878691 conn(0x7f768c0cd9f0 msgr2=0x7f768c0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:21.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.793+0000 7f7694030640 1 -- 172.21.15.88:0/142878691 shutdown_connections 2024-09-21T02:04:21.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.793+0000 7f7694030640 1 -- 172.21.15.88:0/142878691 wait complete. 2024-09-21T02:04:21.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.795+0000 7f7694030640 1 Processor -- start 2024-09-21T02:04:21.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.795+0000 7f7694030640 1 -- start start 2024-09-21T02:04:21.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7694030640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 0x7f768c1f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7694030640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768c0c4dd0 0x7f768c14b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7694030640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768c0c51e0 0x7f768c14b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7694030640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f768c06dcd0 con 0x7f7678004800 2024-09-21T02:04:21.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7694030640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f768c15c640 con 0x7f768c0c4dd0 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7694030640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f768c06de50 con 0x7f768c0c51e0 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 0x7f768c1f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 0x7f768c1f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49696/0 (socket says 172.21.15.88:49696) 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 -- 172.21.15.88:0/1748540594 learned_addr learned my addr 172.21.15.88:0/1748540594 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f76915cb640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768c0c4dd0 0x7f768c14b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f76925cd640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768c0c51e0 0x7f768c14b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 -- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768c0c51e0 msgr2=0x7f768c14b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768c0c51e0 0x7f768c14b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 -- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768c0c4dd0 msgr2=0x7f768c14b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768c0c4dd0 0x7f768c14b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 -- 172.21.15.88:0/1748540594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c1f26a0 con 0x7f7678004800 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f76925cd640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768c0c51e0 0x7f768c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:21.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7691dcc640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 0x7f768c1f2160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7688011e70 tx=0x7f768800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f76827fc640 1 -- 172.21.15.88:0/1748540594 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768801e070 con 0x7f7678004800 2024-09-21T02:04:21.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f76827fc640 1 -- 172.21.15.88:0/1748540594 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7688003d80 con 0x7f7678004800 2024-09-21T02:04:21.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c151830 con 0x7f7678004800 2024-09-21T02:04:21.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.796+0000 7f76827fc640 1 -- 172.21.15.88:0/1748540594 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76880042b0 con 0x7f7678004800 2024-09-21T02:04:21.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.797+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c150d30 con 0x7f7678004800 2024-09-21T02:04:21.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.797+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c08eb80 con 0x7f7678004800 2024-09-21T02:04:21.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.798+0000 7f76827fc640 1 -- 172.21.15.88:0/1748540594 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7688021070 con 0x7f7678004800 2024-09-21T02:04:21.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.799+0000 7f76827fc640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7670078b90 0x7f767007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:21.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.799+0000 7f76827fc640 1 -- 172.21.15.88:0/1748540594 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f768809c470 con 0x7f7678004800 2024-09-21T02:04:21.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.799+0000 7f76915cb640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7670078b90 0x7f767007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:21.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.799+0000 7f76915cb640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7670078b90 0x7f767007b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f767c034000 tx=0x7f767c0046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:21.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:21.802+0000 7f76827fc640 1 -- 172.21.15.88:0/1748540594 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f768c08eb80 con 0x7f7678004800 2024-09-21T02:04:22.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.014+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f768c0cc4a0 con 0x7f7678004800 2024-09-21T02:04:22.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.015+0000 7f76827fc640 1 -- 172.21.15.88:0/1748540594 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) ==== 53+0+73 (secure 0 0 0) 0x7f76880643d0 con 0x7f7678004800 2024-09-21T02:04:22.016 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:04:22.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7670078b90 msgr2=0x7f767007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7670078b90 0x7f767007b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f767c034000 tx=0x7f767c0046c0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 msgr2=0x7f768c1f2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:22.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 0x7f768c1f2160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7688011e70 tx=0x7f768800f790 comp rx=0 tx=0).stop 2024-09-21T02:04:22.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 shutdown_connections 2024-09-21T02:04:22.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7670078b90 0x7f767007b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768c0c51e0 0x7f768c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768c0c4dd0 0x7f768c14b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 --2- 172.21.15.88:0/1748540594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7678004800 0x7f768c1f2160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 >> 172.21.15.88:0/1748540594 conn(0x7f768c0cd9f0 msgr2=0x7f768c14e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 shutdown_connections 2024-09-21T02:04:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22.017+0000 7f7694030640 1 -- 172.21.15.88:0/1748540594 wait complete. 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:04:22.029 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:04:22.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:22.030 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:04:22.045 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42753sec preferred_lft 42753sec 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T02:04:22.046 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:04:22.047 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:04:22.047 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:22.047 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T02:04:22.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:04:22.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2024-09-21T02:04:22.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2024-09-21T02:04:22.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:04:22.047 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T02:04:22.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:22.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:22.242 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:04:22.267 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T02:04:22.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:22.269 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T02:04:22.331 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T02:04:22.332 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:04:22.332 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:04:22.332 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:04:22.332 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:04:22.334 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:22.334 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T02:04:22.334 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:04:22.334 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:04:22.334 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:04:22.334 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T02:04:22.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:22.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:22.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:22.645 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T02:04:22.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:22.660 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-09-21T02:04:22.715 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:22.715 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T02:04:22.715 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:04:22.716 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T02:04:22.716 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T02:04:22.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:22.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:22.833 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T02:04:22.833 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:22.833 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T02:04:22.833 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T02:04:22.833 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:04:22.833 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T02:04:22.833 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T02:04:22.834 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T02:04:22.834 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T02:04:22.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:22.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:22.951 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:04:22.951 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T02:04:22.951 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T02:04:22.951 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:04:22.952 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T02:04:22.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:23.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:23.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:04:23.028 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:23.043 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:04:23.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:04:23.043 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:23.098 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T02:04:23.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:23.098 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T02:04:23.198 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T02:04:23.198 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T02:04:23.198 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T02:04:23.322 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T02:04:23.322 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T02:04:23.323 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T02:04:23.323 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T02:04:23.323 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.195:3300,172.21.15.195:3301" 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.195:6789,172.21.15.195:6790" 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.195:6789,172.21.15.195:6790:/ 2024-09-21T02:04:23.643 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=0ee3c2c3-8406-48b8-a5fb-ec622d976411" will pass to kernel 2024-09-21T02:04:23.644 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T02:04:23.644 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T02:04:23.644 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T02:04:23.644 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T02:04:23.644 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T02:04:23.645 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T02:04:23.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:04:23.646 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:23.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:04:23.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:04:23.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:23.779 INFO:teuthology.orchestra.run.smithi088.stdout:client4741 2024-09-21T02:04:23.779 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:04:23.779 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:04:23.795 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:04:23.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:04:23.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:23.948 INFO:teuthology.orchestra.run.smithi088.stdout:client4741 2024-09-21T02:04:23.949 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:04:24.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:04:25.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:25.285 INFO:teuthology.orchestra.run.smithi088.stdout:client4741 2024-09-21T02:04:25.285 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0ee3c2c3-8406-48b8-a5fb-ec622d976411.client4741/status') 2024-09-21T02:04:25.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:04:25.358 INFO:teuthology.orchestra.run.smithi088.stdout:instance: client.4741 (3)172.21.15.88:0/1691658423 2024-09-21T02:04:25.358 INFO:teuthology.orchestra.run.smithi088.stdout:blocklisted: false 2024-09-21T02:04:25.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records in 2024-09-21T02:04:25.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records out 2024-09-21T02:04:25.358 INFO:teuthology.orchestra.run.smithi088.stderr:70 bytes copied, 2.3881e-05 s, 2.9 MB/s 2024-09-21T02:04:25.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:04:25.367 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-21T02:04:25.406 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-21T02:04:25.406 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:04:25.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.521+0000 7f6ac79a1640 1 -- 172.21.15.88:0/3527736738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 msgr2=0x7f6aa8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.521+0000 7f6ac79a1640 1 --2- 172.21.15.88:0/3527736738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 0x7f6aa8005190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800baf0 tx=0x7f6aa8005830 comp rx=0 tx=0).stop 2024-09-21T02:04:25.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.521+0000 7f6ac5f3e640 1 -- 172.21.15.88:0/3527736738 reap_dead start 2024-09-21T02:04:25.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.521+0000 7f6ac79a1640 1 -- 172.21.15.88:0/3527736738 shutdown_connections 2024-09-21T02:04:25.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.521+0000 7f6ac79a1640 1 -- 172.21.15.88:0/3527736738 >> 172.21.15.88:0/3527736738 conn(0x7f6ac00cd9f0 msgr2=0x7f6ac00cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.521+0000 7f6ac79a1640 1 -- 172.21.15.88:0/3527736738 shutdown_connections 2024-09-21T02:04:25.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.521+0000 7f6ac79a1640 1 -- 172.21.15.88:0/3527736738 wait complete. 2024-09-21T02:04:25.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.522+0000 7f6ac79a1640 1 Processor -- start 2024-09-21T02:04:25.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.522+0000 7f6ac79a1640 1 -- start start 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac79a1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 0x7f6ac01f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac79a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ac00c68a0 0x7f6ac0157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac79a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ac00c6cb0 0x7f6ac01552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac79a1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac015c640 con 0x7f6aa8004d40 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac79a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac006d470 con 0x7f6ac00c6cb0 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac79a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac006d5f0 con 0x7f6ac00c68a0 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 0x7f6ac01f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 0x7f6ac01f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49712/0 (socket says 172.21.15.88:49712) 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 -- 172.21.15.88:0/4208079057 learned_addr learned my addr 172.21.15.88:0/4208079057 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac4f3c640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ac00c68a0 0x7f6ac0157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac5f3e640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ac00c6cb0 0x7f6ac01552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 -- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ac00c68a0 msgr2=0x7f6ac0157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ac00c68a0 0x7f6ac0157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 -- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ac00c6cb0 msgr2=0x7f6ac01552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ac00c6cb0 0x7f6ac01552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 -- 172.21.15.88:0/4208079057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac01f5dd0 con 0x7f6aa8004d40 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac573d640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 0x7f6ac01f1c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009a30 tx=0x7f6ab0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6abdffb640 1 -- 172.21.15.88:0/4208079057 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0004d20 con 0x7f6aa8004d40 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6abdffb640 1 -- 172.21.15.88:0/4208079057 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ab000faf0 con 0x7f6aa8004d40 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac01f6070 con 0x7f6aa8004d40 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.524+0000 7f6abdffb640 1 -- 172.21.15.88:0/4208079057 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0005020 con 0x7f6aa8004d40 2024-09-21T02:04:25.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.525+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac0158dd0 con 0x7f6aa8004d40 2024-09-21T02:04:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.525+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac015f970 con 0x7f6aa8004d40 2024-09-21T02:04:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.526+0000 7f6abdffb640 1 -- 172.21.15.88:0/4208079057 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ab00051c0 con 0x7f6aa8004d40 2024-09-21T02:04:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.527+0000 7f6abdffb640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6aac078b90 0x7f6aac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.527+0000 7f6abdffb640 1 -- 172.21.15.88:0/4208079057 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6ab009ba60 con 0x7f6aa8004d40 2024-09-21T02:04:25.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.527+0000 7f6ac4f3c640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6aac078b90 0x7f6aac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.528+0000 7f6ac4f3c640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6aac078b90 0x7f6aac07b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4001880 tx=0x7f6ab4006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.531+0000 7f6abdffb640 1 -- 172.21.15.88:0/4208079057 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ab0063970 con 0x7f6aa8004d40 2024-09-21T02:04:25.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.710+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ac00c4880 con 0x7f6aa8004d40 2024-09-21T02:04:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.712+0000 7f6abdffb640 1 -- 172.21.15.88:0/4208079057 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f6ab00683a0 con 0x7f6aa8004d40 2024-09-21T02:04:25.713 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:25.713 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":21,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:03:59.066086+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6aac078b90 msgr2=0x7f6aac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6aac078b90 0x7f6aac07b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4001880 tx=0x7f6ab4006040 comp rx=0 tx=0).stop 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 msgr2=0x7f6ac01f1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 0x7f6ac01f1c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009a30 tx=0x7f6ab0009ef0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 shutdown_connections 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6aac078b90 0x7f6aac07b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6ac00c6cb0 0x7f6ac01552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ac00c68a0 0x7f6ac0157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 --2- 172.21.15.88:0/4208079057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa8004d40 0x7f6ac01f1c10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 >> 172.21.15.88:0/4208079057 conn(0x7f6ac00cd9f0 msgr2=0x7f6ac015f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 shutdown_connections 2024-09-21T02:04:25.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.714+0000 7f6ac79a1640 1 -- 172.21.15.88:0/4208079057 wait complete. 2024-09-21T02:04:25.726 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:03:44.834536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:03:55.301344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:03:56.381370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:04:25.726 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:04:25.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 -- 172.21.15.88:0/1383391639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 msgr2=0x7faaa80c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 --2- 172.21.15.88:0/1383391639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa80c3e70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faaa80738d0 tx=0x7faaa002aac0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 -- 172.21.15.88:0/1383391639 shutdown_connections 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 --2- 172.21.15.88:0/1383391639 >> v2:172.21.15.195:3300/0 conn(0x7faaa80c5d90 0x7faaa80726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 --2- 172.21.15.88:0/1383391639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa80c3e70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 --2- 172.21.15.88:0/1383391639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa8071f60 0x7faaa815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 -- 172.21.15.88:0/1383391639 >> 172.21.15.88:0/1383391639 conn(0x7faaa8068d00 msgr2=0x7faaa8069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 -- 172.21.15.88:0/1383391639 shutdown_connections 2024-09-21T02:04:25.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.840+0000 7faaae50b640 1 -- 172.21.15.88:0/1383391639 wait complete. 2024-09-21T02:04:25.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.842+0000 7faaae50b640 1 Processor -- start 2024-09-21T02:04:25.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.842+0000 7faaae50b640 1 -- start start 2024-09-21T02:04:25.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.843+0000 7faaae50b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80c5d90 0x7faaa81ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.843+0000 7faaae50b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa81541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.843+0000 7faaae50b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80c6290 0x7faaa8149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.843+0000 7faaae50b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8153cb0 con 0x7faaa80c8e60 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.843+0000 7faaae50b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8159dc0 con 0x7faaa80c6290 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.843+0000 7faaae50b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7faaa8153e30 con 0x7faaa80c5d90 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.843+0000 7faaa77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa81541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa81541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49722/0 (socket says 172.21.15.88:49722) 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 -- 172.21.15.88:0/1759375033 learned_addr learned my addr 172.21.15.88:0/1759375033 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaacaa8640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80c6290 0x7faaa8149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa7fff640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80c5d90 0x7faaa81ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 -- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80c5d90 msgr2=0x7faaa81ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80c5d90 0x7faaa81ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 -- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80c6290 msgr2=0x7faaa8149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80c6290 0x7faaa8149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 -- 172.21.15.88:0/1759375033 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa80c9790 con 0x7faaa80c8e60 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa77fe640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa81541f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa9000d680 tx=0x7faa9000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa4ff9640 1 -- 172.21.15.88:0/1759375033 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90017070 con 0x7faaa80c8e60 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa4ff9640 1 -- 172.21.15.88:0/1759375033 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa90009b10 con 0x7faaa80c8e60 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaa4ff9640 1 -- 172.21.15.88:0/1759375033 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90010460 con 0x7faaa80c8e60 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa80c9a70 con 0x7faaa80c8e60 2024-09-21T02:04:25.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.844+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa8154730 con 0x7faaa80c8e60 2024-09-21T02:04:25.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.845+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa80cbee0 con 0x7faaa80c8e60 2024-09-21T02:04:25.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.846+0000 7faaa4ff9640 1 -- 172.21.15.88:0/1759375033 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa9001a070 con 0x7faaa80c8e60 2024-09-21T02:04:25.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.847+0000 7faaa4ff9640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faa88078bc0 0x7faa8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:25.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.847+0000 7faaa4ff9640 1 -- 172.21.15.88:0/1759375033 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faa9009ae40 con 0x7faaa80c8e60 2024-09-21T02:04:25.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.847+0000 7faaa7fff640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faa88078bc0 0x7faa8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:25.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.847+0000 7faaa7fff640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faa88078bc0 0x7faa8807b0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa9800daa0 tx=0x7faa98005d70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:25.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:25.851+0000 7faaa4ff9640 1 -- 172.21.15.88:0/1759375033 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faa90063290 con 0x7faaa80c8e60 2024-09-21T02:04:26.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.024+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faaa8071f60 con 0x7faaa80c8e60 2024-09-21T02:04:26.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.025+0000 7faaa4ff9640 1 -- 172.21.15.88:0/1759375033 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7faa90062b90 con 0x7faaa80c8e60 2024-09-21T02:04:26.026 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T02:04:26.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faa88078bc0 msgr2=0x7faa8807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faa88078bc0 0x7faa8807b0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa9800daa0 tx=0x7faa98005d70 comp rx=0 tx=0).stop 2024-09-21T02:04:26.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 msgr2=0x7faaa81541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa81541f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa9000d680 tx=0x7faa9000db40 comp rx=0 tx=0).stop 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 shutdown_connections 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7faa88078bc0 0x7faa8807b0d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faaa80c6290 0x7faaa8149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faaa80c8e60 0x7faaa81541f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 --2- 172.21.15.88:0/1759375033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faaa80c5d90 0x7faaa81ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 >> 172.21.15.88:0/1759375033 conn(0x7faaa8068d00 msgr2=0x7faaa80c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 shutdown_connections 2024-09-21T02:04:26.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.027+0000 7faaae50b640 1 -- 172.21.15.88:0/1759375033 wait complete. 2024-09-21T02:04:26.039 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.154+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/2825132980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c5d90 msgr2=0x7fa7cc0726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.154+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/2825132980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c5d90 0x7fa7cc0726d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc005fd0 tx=0x7fa7bc02a5c0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.155+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/2825132980 shutdown_connections 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.155+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/2825132980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c5d90 0x7fa7cc0726d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.155+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/2825132980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7cc071f60 0x7fa7cc15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.155+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/2825132980 >> 172.21.15.88:0/2825132980 conn(0x7fa7cc068d00 msgr2=0x7fa7cc069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.155+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/2825132980 shutdown_connections 2024-09-21T02:04:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.155+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/2825132980 wait complete. 2024-09-21T02:04:26.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.156+0000 7fa7d3ba1640 1 Processor -- start 2024-09-21T02:04:26.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.156+0000 7fa7d3ba1640 1 -- start start 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.157+0000 7fa7d3ba1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc0c5d90 0x7fa7cc152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.157+0000 7fa7d3ba1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c8e60 0x7fa7cc0c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.157+0000 7fa7d3ba1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7cc146780 0x7fa7cc146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.157+0000 7fa7d3ba1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc153cb0 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.157+0000 7fa7d3ba1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc159dc0 con 0x7fa7cc0c5d90 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.157+0000 7fa7d3ba1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc153e30 con 0x7fa7cc146780 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c8e60 0x7fa7cc0c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c8e60 0x7fa7cc0c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49726/0 (socket says 172.21.15.88:49726) 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 -- 172.21.15.88:0/135468490 learned_addr learned my addr 172.21.15.88:0/135468490 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d193d640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc0c5d90 0x7fa7cc152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d213e640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7cc146780 0x7fa7cc146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 -- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7cc146780 msgr2=0x7fa7cc146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7cc146780 0x7fa7cc146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 -- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc0c5d90 msgr2=0x7fa7cc152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc0c5d90 0x7fa7cc152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 -- 172.21.15.88:0/135468490 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc14f340 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d113c640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c8e60 0x7fa7cc0c9a60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc006010 tx=0x7fa7bc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7c27fc640 1 -- 172.21.15.88:0/135468490 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc03e030 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7c27fc640 1 -- 172.21.15.88:0/135468490 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7bc0047a0 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7c27fc640 1 -- 172.21.15.88:0/135468490 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc035040 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc14f620 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.158+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc14c910 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.159+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc0c46c0 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.160+0000 7fa7c27fc640 1 -- 172.21.15.88:0/135468490 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa7bc041030 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.161+0000 7fa7c27fc640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa7b0078b90 0x7fa7b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.161+0000 7fa7c27fc640 1 -- 172.21.15.88:0/135468490 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa7bc0bb570 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.161+0000 7fa7d193d640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa7b0078b90 0x7fa7b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.161+0000 7fa7d193d640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa7b0078b90 0x7fa7b007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005c40 tx=0x7fa7c800a530 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.165+0000 7fa7c27fc640 1 -- 172.21.15.88:0/135468490 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa7bc083400 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.343+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa7cc0c9320 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.344+0000 7fa7c27fc640 1 -- 172.21.15.88:0/135468490 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fa7bc087e30 con 0x7fa7cc0c8e60 2024-09-21T02:04:26.345 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa7b0078b90 msgr2=0x7fa7b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa7b0078b90 0x7fa7b007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8005c40 tx=0x7fa7c800a530 comp rx=0 tx=0).stop 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c8e60 msgr2=0x7fa7cc0c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c8e60 0x7fa7cc0c9a60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc006010 tx=0x7fa7bc0039e0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 shutdown_connections 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa7b0078b90 0x7fa7b007b0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7cc146780 0x7fa7cc146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc0c8e60 0x7fa7cc0c9a60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 --2- 172.21.15.88:0/135468490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7cc0c5d90 0x7fa7cc152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 >> 172.21.15.88:0/135468490 conn(0x7fa7cc068d00 msgr2=0x7fa7cc14bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 shutdown_connections 2024-09-21T02:04:26.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.346+0000 7fa7d3ba1640 1 -- 172.21.15.88:0/135468490 wait complete. 2024-09-21T02:04:26.358 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T02:04:26.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 -- 172.21.15.88:0/1094274730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f503015b980 msgr2=0x7f50300b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 --2- 172.21.15.88:0/1094274730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f503015b980 0x7f50300b4c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f503006da20 tx=0x7f502002a560 comp rx=0 tx=0).stop 2024-09-21T02:04:26.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 -- 172.21.15.88:0/1094274730 shutdown_connections 2024-09-21T02:04:26.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 --2- 172.21.15.88:0/1094274730 >> v2:172.21.15.195:3300/0 conn(0x7f50300cb990 0x7f50300cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 --2- 172.21.15.88:0/1094274730 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50300b51f0 0x7f50300b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 --2- 172.21.15.88:0/1094274730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f503015b980 0x7f50300b4c10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 -- 172.21.15.88:0/1094274730 >> 172.21.15.88:0/1094274730 conn(0x7f50300cda90 msgr2=0x7f50300cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 -- 172.21.15.88:0/1094274730 shutdown_connections 2024-09-21T02:04:26.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.472+0000 7f5036a4d640 1 -- 172.21.15.88:0/1094274730 wait complete. 2024-09-21T02:04:26.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.474+0000 7f5036a4d640 1 Processor -- start 2024-09-21T02:04:26.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.474+0000 7f5036a4d640 1 -- start start 2024-09-21T02:04:26.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.475+0000 7f5036a4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50300b51f0 0x7f5030149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.475+0000 7f5036a4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50300cb990 0x7f50301f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.475+0000 7f5036a4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503015b980 0x7f50300c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.475+0000 7f5036a4d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f50300b46f0 con 0x7f50300cb990 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.475+0000 7f5036a4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f50300b4410 con 0x7f503015b980 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.475+0000 7f5036a4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f503006cd60 con 0x7f50300b51f0 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.475+0000 7f502f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50300cb990 0x7f50301f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50300cb990 0x7f50301f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49742/0 (socket says 172.21.15.88:49742) 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 -- 172.21.15.88:0/2905670964 learned_addr learned my addr 172.21.15.88:0/2905670964 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502ffff640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50300b51f0 0x7f5030149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f5034fea640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503015b980 0x7f50300c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 -- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50300b51f0 msgr2=0x7f5030149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50300b51f0 0x7f5030149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 -- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503015b980 msgr2=0x7f50300c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503015b980 0x7f50300c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 -- 172.21.15.88:0/2905670964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50300c74e0 con 0x7f50300cb990 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502f7fe640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50300cb990 0x7f50301f5b10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f502400d6d0 tx=0x7f502400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502cff9640 1 -- 172.21.15.88:0/2905670964 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024017070 con 0x7f50300cb990 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502cff9640 1 -- 172.21.15.88:0/2905670964 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5024009ab0 con 0x7f50300cb990 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5030149c80 con 0x7f50300cb990 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f502cff9640 1 -- 172.21.15.88:0/2905670964 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024010460 con 0x7f50300cb990 2024-09-21T02:04:26.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.476+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50301e41f0 con 0x7f50300cb990 2024-09-21T02:04:26.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.477+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50300ca770 con 0x7f50300cb990 2024-09-21T02:04:26.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.478+0000 7f502cff9640 1 -- 172.21.15.88:0/2905670964 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f502401a070 con 0x7f50300cb990 2024-09-21T02:04:26.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.478+0000 7f502cff9640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f501c078b90 0x7f501c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.479+0000 7f502cff9640 1 -- 172.21.15.88:0/2905670964 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f502409ae00 con 0x7f50300cb990 2024-09-21T02:04:26.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.479+0000 7f502ffff640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f501c078b90 0x7f501c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.479+0000 7f502ffff640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f501c078b90 0x7f501c07b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5020033000 tx=0x7f5020004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:26.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.483+0000 7f502cff9640 1 -- 172.21.15.88:0/2905670964 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50300ca770 con 0x7f50300cb990 2024-09-21T02:04:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.657+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f503015c0d0 con 0x7f50300cb990 2024-09-21T02:04:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.658+0000 7f502cff9640 1 -- 172.21.15.88:0/2905670964 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f5030149c80 con 0x7f50300cb990 2024-09-21T02:04:26.659 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-09-21T02:04:26.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f501c078b90 msgr2=0x7f501c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f501c078b90 0x7f501c07b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5020033000 tx=0x7f5020004730 comp rx=0 tx=0).stop 2024-09-21T02:04:26.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50300cb990 msgr2=0x7f50301f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50300cb990 0x7f50301f5b10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f502400d6d0 tx=0x7f502400db90 comp rx=0 tx=0).stop 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 shutdown_connections 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f501c078b90 0x7f501c07b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f503015b980 0x7f50300c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50300cb990 0x7f50301f5b10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 --2- 172.21.15.88:0/2905670964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50300b51f0 0x7f5030149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 >> 172.21.15.88:0/2905670964 conn(0x7f50300cda90 msgr2=0x7f5030155670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 shutdown_connections 2024-09-21T02:04:26.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.660+0000 7f5036a4d640 1 -- 172.21.15.88:0/2905670964 wait complete. 2024-09-21T02:04:26.672 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-21T02:04:26.672 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.784+0000 7f18215e6640 1 -- 172.21.15.88:0/1375865264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1804004800 msgr2=0x7f1804004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.784+0000 7f18215e6640 1 --2- 172.21.15.88:0/1375865264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1804004800 0x7f1804004c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f181c0c5ac0 tx=0x7f1804005830 comp rx=0 tx=0).stop 2024-09-21T02:04:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.784+0000 7f18215e6640 1 -- 172.21.15.88:0/1375865264 shutdown_connections 2024-09-21T02:04:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.784+0000 7f18215e6640 1 --2- 172.21.15.88:0/1375865264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1804004800 0x7f1804004c10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.784+0000 7f18215e6640 1 -- 172.21.15.88:0/1375865264 >> 172.21.15.88:0/1375865264 conn(0x7f181c073210 msgr2=0x7f181c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.784+0000 7f18215e6640 1 -- 172.21.15.88:0/1375865264 shutdown_connections 2024-09-21T02:04:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.784+0000 7f18215e6640 1 -- 172.21.15.88:0/1375865264 wait complete. 2024-09-21T02:04:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.786+0000 7f18215e6640 1 Processor -- start 2024-09-21T02:04:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.786+0000 7f18215e6640 1 -- start start 2024-09-21T02:04:26.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f18215e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1804004800 0x7f181c15c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f18215e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f181c0c63a0 0x7f181c0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f18215e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181c1e5ef0 0x7f181c14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f18215e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f181c070ae0 con 0x7f181c0c63a0 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f18215e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f181c071fc0 con 0x7f181c1e5ef0 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f18215e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f181c070dc0 con 0x7f1804004800 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f181c0c63a0 0x7f181c0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f181c0c63a0 0x7f181c0c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49768/0 (socket says 172.21.15.88:49768) 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 -- 172.21.15.88:0/2746213399 learned_addr learned my addr 172.21.15.88:0/2746213399 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181affd640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1804004800 0x7f181c15c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181b7fe640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181c1e5ef0 0x7f181c14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 -- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1804004800 msgr2=0x7f181c15c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1804004800 0x7f181c15c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 -- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181c1e5ef0 msgr2=0x7f181c14be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181c1e5ef0 0x7f181c14be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 -- 172.21.15.88:0/2746213399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c0c7080 con 0x7f181c0c63a0 2024-09-21T02:04:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.787+0000 7f181a7fc640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f181c0c63a0 0x7f181c0c6b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1810005e00 tx=0x7f1810004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.788+0000 7f18037fe640 1 -- 172.21.15.88:0/2746213399 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810041020 con 0x7f181c0c63a0 2024-09-21T02:04:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.788+0000 7f18037fe640 1 -- 172.21.15.88:0/2746213399 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f181002ba40 con 0x7f181c0c63a0 2024-09-21T02:04:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.788+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c14cc00 con 0x7f181c0c63a0 2024-09-21T02:04:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.788+0000 7f18037fe640 1 -- 172.21.15.88:0/2746213399 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810004e50 con 0x7f181c0c63a0 2024-09-21T02:04:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.788+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c15a900 con 0x7f181c0c63a0 2024-09-21T02:04:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.789+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c0ca8c0 con 0x7f181c0c63a0 2024-09-21T02:04:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.789+0000 7f18037fe640 1 -- 172.21.15.88:0/2746213399 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1810044020 con 0x7f181c0c63a0 2024-09-21T02:04:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.790+0000 7f18037fe640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1808078b90 0x7f180807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.790+0000 7f18037fe640 1 -- 172.21.15.88:0/2746213399 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f18100bbd90 con 0x7f181c0c63a0 2024-09-21T02:04:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.790+0000 7f181affd640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1808078b90 0x7f180807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.791+0000 7f181affd640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1808078b90 0x7f180807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f180c000f80 tx=0x7f180c006e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:26.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.794+0000 7f18037fe640 1 -- 172.21.15.88:0/2746213399 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f181004b030 con 0x7f181c0c63a0 2024-09-21T02:04:26.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.968+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f181c15d8f0 con 0x7f1808078b90 2024-09-21T02:04:26.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.973+0000 7f18037fe640 1 -- 172.21.15.88:0/2746213399 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251884 (secure 0 0 0) 0x7f181c15d8f0 con 0x7f1808078b90 2024-09-21T02:04:26.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:26.977 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1808078b90 msgr2=0x7f180807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1808078b90 0x7f180807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f180c000f80 tx=0x7f180c006e90 comp rx=0 tx=0).stop 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f181c0c63a0 msgr2=0x7f181c0c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f181c0c63a0 0x7f181c0c6b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1810005e00 tx=0x7f1810004a50 comp rx=0 tx=0).stop 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 shutdown_connections 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1808078b90 0x7f180807b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181c1e5ef0 0x7f181c14be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f181c0c63a0 0x7f181c0c6b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 --2- 172.21.15.88:0/2746213399 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1804004800 0x7f181c15c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 >> 172.21.15.88:0/2746213399 conn(0x7f181c073210 msgr2=0x7f181c14d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 shutdown_connections 2024-09-21T02:04:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:26.976+0000 7f18215e6640 1 -- 172.21.15.88:0/2746213399 wait complete. 2024-09-21T02:04:26.989 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T02:04:26.824932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596102,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":229,"num_write_kb":1669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239944,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734776,"statfs":{"total":773094113280,"available":772848410624,"internally_reserved":0,"allocated":3022848,"data_stored":1774180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1918,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074727+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:03:59.074727+0000","last_peered":"2024-09-21T02:03:59.074727+0000","last_clean":"2024-09-21T02:03:59.074727+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:03:59.074727+0000","last_undegraded":"2024-09-21T02:03:59.074727+0000","last_fullsized":"2024-09-21T02:03:59.074727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073213+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:03:59.073213+0000","last_peered":"2024-09-21T02:03:59.073213+0000","last_clean":"2024-09-21T02:03:59.073213+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:03:59.073213+0000","last_undegraded":"2024-09-21T02:03:59.073213+0000","last_fullsized":"2024-09-21T02:03:59.073213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836539+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:03:59.836539+0000","last_peered":"2024-09-21T02:03:59.836539+0000","last_clean":"2024-09-21T02:03:59.836539+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:03:59.836539+0000","last_undegraded":"2024-09-21T02:03:59.836539+0000","last_fullsized":"2024-09-21T02:03:59.836539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075220+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:03:59.075220+0000","last_peered":"2024-09-21T02:03:59.075220+0000","last_clean":"2024-09-21T02:03:59.075220+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:03:59.075220+0000","last_undegraded":"2024-09-21T02:03:59.075220+0000","last_fullsized":"2024-09-21T02:03:59.075220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074990+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:03:59.074990+0000","last_peered":"2024-09-21T02:03:59.074990+0000","last_clean":"2024-09-21T02:03:59.074990+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:03:59.074990+0000","last_undegraded":"2024-09-21T02:03:59.074990+0000","last_fullsized":"2024-09-21T02:03:59.074990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074223+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:03:59.074223+0000","last_peered":"2024-09-21T02:03:59.074223+0000","last_clean":"2024-09-21T02:03:59.074223+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:03:59.074223+0000","last_undegraded":"2024-09-21T02:03:59.074223+0000","last_fullsized":"2024-09-21T02:03:59.074223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074095+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:03:59.074095+0000","last_peered":"2024-09-21T02:03:59.074095+0000","last_clean":"2024-09-21T02:03:59.074095+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:03:59.074095+0000","last_undegraded":"2024-09-21T02:03:59.074095+0000","last_fullsized":"2024-09-21T02:03:59.074095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835347+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:03:59.835347+0000","last_peered":"2024-09-21T02:03:59.835347+0000","last_clean":"2024-09-21T02:03:59.835347+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:03:59.835347+0000","last_undegraded":"2024-09-21T02:03:59.835347+0000","last_fullsized":"2024-09-21T02:03:59.835347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074637+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:03:59.074637+0000","last_peered":"2024-09-21T02:03:59.074637+0000","last_clean":"2024-09-21T02:03:59.074637+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:03:59.074637+0000","last_undegraded":"2024-09-21T02:03:59.074637+0000","last_fullsized":"2024-09-21T02:03:59.074637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835966+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:03:59.835966+0000","last_peered":"2024-09-21T02:03:59.835966+0000","last_clean":"2024-09-21T02:03:59.835966+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:03:59.835966+0000","last_undegraded":"2024-09-21T02:03:59.835966+0000","last_fullsized":"2024-09-21T02:03:59.835966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835587+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:03:59.835587+0000","last_peered":"2024-09-21T02:03:59.835587+0000","last_clean":"2024-09-21T02:03:59.835587+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:03:59.835587+0000","last_undegraded":"2024-09-21T02:03:59.835587+0000","last_fullsized":"2024-09-21T02:03:59.835587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781637+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:03:59.781637+0000","last_peered":"2024-09-21T02:03:59.781637+0000","last_clean":"2024-09-21T02:03:59.781637+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:03:59.781637+0000","last_undegraded":"2024-09-21T02:03:59.781637+0000","last_fullsized":"2024-09-21T02:03:59.781637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835918+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:03:59.835918+0000","last_peered":"2024-09-21T02:03:59.835918+0000","last_clean":"2024-09-21T02:03:59.835918+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:03:59.835918+0000","last_undegraded":"2024-09-21T02:03:59.835918+0000","last_fullsized":"2024-09-21T02:03:59.835918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836186+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:03:59.836186+0000","last_peered":"2024-09-21T02:03:59.836186+0000","last_clean":"2024-09-21T02:03:59.836186+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:03:59.836186+0000","last_undegraded":"2024-09-21T02:03:59.836186+0000","last_fullsized":"2024-09-21T02:03:59.836186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072763+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:03:59.072763+0000","last_peered":"2024-09-21T02:03:59.072763+0000","last_clean":"2024-09-21T02:03:59.072763+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:03:59.072763+0000","last_undegraded":"2024-09-21T02:03:59.072763+0000","last_fullsized":"2024-09-21T02:03:59.072763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075548+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:03:59.075548+0000","last_peered":"2024-09-21T02:03:59.075548+0000","last_clean":"2024-09-21T02:03:59.075548+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:03:59.075548+0000","last_undegraded":"2024-09-21T02:03:59.075548+0000","last_fullsized":"2024-09-21T02:03:59.075548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075910+0000","last_change":"2024-09-21T02:03:58.074570+0000","last_active":"2024-09-21T02:03:59.075910+0000","last_peered":"2024-09-21T02:03:59.075910+0000","last_clean":"2024-09-21T02:03:59.075910+0000","last_became_active":"2024-09-21T02:03:58.074271+0000","last_became_peered":"2024-09-21T02:03:58.074271+0000","last_unstale":"2024-09-21T02:03:59.075910+0000","last_undegraded":"2024-09-21T02:03:59.075910+0000","last_fullsized":"2024-09-21T02:03:59.075910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.292792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835423+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:03:59.835423+0000","last_peered":"2024-09-21T02:03:59.835423+0000","last_clean":"2024-09-21T02:03:59.835423+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:03:59.835423+0000","last_undegraded":"2024-09-21T02:03:59.835423+0000","last_fullsized":"2024-09-21T02:03:59.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076132+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:03:59.076132+0000","last_peered":"2024-09-21T02:03:59.076132+0000","last_clean":"2024-09-21T02:03:59.076132+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:03:59.076132+0000","last_undegraded":"2024-09-21T02:03:59.076132+0000","last_fullsized":"2024-09-21T02:03:59.076132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835510+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:03:59.835510+0000","last_peered":"2024-09-21T02:03:59.835510+0000","last_clean":"2024-09-21T02:03:59.835510+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:03:59.835510+0000","last_undegraded":"2024-09-21T02:03:59.835510+0000","last_fullsized":"2024-09-21T02:03:59.835510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836526+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:03:59.836526+0000","last_peered":"2024-09-21T02:03:59.836526+0000","last_clean":"2024-09-21T02:03:59.836526+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:03:59.836526+0000","last_undegraded":"2024-09-21T02:03:59.836526+0000","last_fullsized":"2024-09-21T02:03:59.836526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075251+0000","last_change":"2024-09-21T02:03:58.076378+0000","last_active":"2024-09-21T02:03:59.075251+0000","last_peered":"2024-09-21T02:03:59.075251+0000","last_clean":"2024-09-21T02:03:59.075251+0000","last_became_active":"2024-09-21T02:03:58.076075+0000","last_became_peered":"2024-09-21T02:03:58.076075+0000","last_unstale":"2024-09-21T02:03:59.075251+0000","last_undegraded":"2024-09-21T02:03:59.075251+0000","last_fullsized":"2024-09-21T02:03:59.075251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:54.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835596+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:03:59.835596+0000","last_peered":"2024-09-21T02:03:59.835596+0000","last_clean":"2024-09-21T02:03:59.835596+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:03:59.835596+0000","last_undegraded":"2024-09-21T02:03:59.835596+0000","last_fullsized":"2024-09-21T02:03:59.835596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074571+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:03:59.074571+0000","last_peered":"2024-09-21T02:03:59.074571+0000","last_clean":"2024-09-21T02:03:59.074571+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:03:59.074571+0000","last_undegraded":"2024-09-21T02:03:59.074571+0000","last_fullsized":"2024-09-21T02:03:59.074571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836762+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:03:59.836762+0000","last_peered":"2024-09-21T02:03:59.836762+0000","last_clean":"2024-09-21T02:03:59.836762+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:03:59.836762+0000","last_undegraded":"2024-09-21T02:03:59.836762+0000","last_fullsized":"2024-09-21T02:03:59.836762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835955+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:03:59.835955+0000","last_peered":"2024-09-21T02:03:59.835955+0000","last_clean":"2024-09-21T02:03:59.835955+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:03:59.835955+0000","last_undegraded":"2024-09-21T02:03:59.835955+0000","last_fullsized":"2024-09-21T02:03:59.835955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073053+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:03:59.073053+0000","last_peered":"2024-09-21T02:03:59.073053+0000","last_clean":"2024-09-21T02:03:59.073053+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:03:59.073053+0000","last_undegraded":"2024-09-21T02:03:59.073053+0000","last_fullsized":"2024-09-21T02:03:59.073053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836383+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:03:59.836383+0000","last_peered":"2024-09-21T02:03:59.836383+0000","last_clean":"2024-09-21T02:03:59.836383+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:03:59.836383+0000","last_undegraded":"2024-09-21T02:03:59.836383+0000","last_fullsized":"2024-09-21T02:03:59.836383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836861+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:03:59.836861+0000","last_peered":"2024-09-21T02:03:59.836861+0000","last_clean":"2024-09-21T02:03:59.836861+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:03:59.836861+0000","last_undegraded":"2024-09-21T02:03:59.836861+0000","last_fullsized":"2024-09-21T02:03:59.836861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075554+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:03:59.075554+0000","last_peered":"2024-09-21T02:03:59.075554+0000","last_clean":"2024-09-21T02:03:59.075554+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:03:59.075554+0000","last_undegraded":"2024-09-21T02:03:59.075554+0000","last_fullsized":"2024-09-21T02:03:59.075554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835391+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:03:59.835391+0000","last_peered":"2024-09-21T02:03:59.835391+0000","last_clean":"2024-09-21T02:03:59.835391+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:03:59.835391+0000","last_undegraded":"2024-09-21T02:03:59.835391+0000","last_fullsized":"2024-09-21T02:03:59.835391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835439+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:03:59.835439+0000","last_peered":"2024-09-21T02:03:59.835439+0000","last_clean":"2024-09-21T02:03:59.835439+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:03:59.835439+0000","last_undegraded":"2024-09-21T02:03:59.835439+0000","last_fullsized":"2024-09-21T02:03:59.835439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781525+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:03:59.781525+0000","last_peered":"2024-09-21T02:03:59.781525+0000","last_clean":"2024-09-21T02:03:59.781525+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:03:59.781525+0000","last_undegraded":"2024-09-21T02:03:59.781525+0000","last_fullsized":"2024-09-21T02:03:59.781525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107531+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:04:08.107531+0000","last_peered":"2024-09-21T02:04:08.107531+0000","last_clean":"2024-09-21T02:04:08.107531+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:04:08.107531+0000","last_undegraded":"2024-09-21T02:04:08.107531+0000","last_fullsized":"2024-09-21T02:04:08.107531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836856+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:03:59.836856+0000","last_peered":"2024-09-21T02:03:59.836856+0000","last_clean":"2024-09-21T02:03:59.836856+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:03:59.836856+0000","last_undegraded":"2024-09-21T02:03:59.836856+0000","last_fullsized":"2024-09-21T02:03:59.836856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.137774+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:04:15.137774+0000","last_peered":"2024-09-21T02:04:15.137774+0000","last_clean":"2024-09-21T02:04:15.137774+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:04:15.137774+0000","last_undegraded":"2024-09-21T02:04:15.137774+0000","last_fullsized":"2024-09-21T02:04:15.137774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.137911+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:04:15.137911+0000","last_peered":"2024-09-21T02:04:15.137911+0000","last_clean":"2024-09-21T02:04:15.137911+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:04:15.137911+0000","last_undegraded":"2024-09-21T02:04:15.137911+0000","last_fullsized":"2024-09-21T02:04:15.137911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835573+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:03:59.835573+0000","last_peered":"2024-09-21T02:03:59.835573+0000","last_clean":"2024-09-21T02:03:59.835573+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:03:59.835573+0000","last_undegraded":"2024-09-21T02:03:59.835573+0000","last_fullsized":"2024-09-21T02:03:59.835573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074720+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:03:59.074720+0000","last_peered":"2024-09-21T02:03:59.074720+0000","last_clean":"2024-09-21T02:03:59.074720+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:03:59.074720+0000","last_undegraded":"2024-09-21T02:03:59.074720+0000","last_fullsized":"2024-09-21T02:03:59.074720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072701+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:03:59.072701+0000","last_peered":"2024-09-21T02:03:59.072701+0000","last_clean":"2024-09-21T02:03:59.072701+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:03:59.072701+0000","last_undegraded":"2024-09-21T02:03:59.072701+0000","last_fullsized":"2024-09-21T02:03:59.072701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.108080+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:04:08.108080+0000","last_peered":"2024-09-21T02:04:08.108080+0000","last_clean":"2024-09-21T02:04:08.108080+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:04:08.108080+0000","last_undegraded":"2024-09-21T02:04:08.108080+0000","last_fullsized":"2024-09-21T02:04:08.108080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074748+0000","last_change":"2024-09-21T02:03:58.075434+0000","last_active":"2024-09-21T02:03:59.074748+0000","last_peered":"2024-09-21T02:03:59.074748+0000","last_clean":"2024-09-21T02:03:59.074748+0000","last_became_active":"2024-09-21T02:03:58.075130+0000","last_became_peered":"2024-09-21T02:03:58.075130+0000","last_unstale":"2024-09-21T02:03:59.074748+0000","last_undegraded":"2024-09-21T02:03:59.074748+0000","last_fullsized":"2024-09-21T02:03:59.074748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:56.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107586+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:04:08.107586+0000","last_peered":"2024-09-21T02:04:08.107586+0000","last_clean":"2024-09-21T02:04:08.107586+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:04:08.107586+0000","last_undegraded":"2024-09-21T02:04:08.107586+0000","last_fullsized":"2024-09-21T02:04:08.107586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836482+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:03:59.836482+0000","last_peered":"2024-09-21T02:03:59.836482+0000","last_clean":"2024-09-21T02:03:59.836482+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:03:59.836482+0000","last_undegraded":"2024-09-21T02:03:59.836482+0000","last_fullsized":"2024-09-21T02:03:59.836482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070253+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:04:03.070253+0000","last_peered":"2024-09-21T02:04:03.070253+0000","last_clean":"2024-09-21T02:04:03.070253+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:04:03.070253+0000","last_undegraded":"2024-09-21T02:04:03.070253+0000","last_fullsized":"2024-09-21T02:04:03.070253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781695+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:03:59.781695+0000","last_peered":"2024-09-21T02:03:59.781695+0000","last_clean":"2024-09-21T02:03:59.781695+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:03:59.781695+0000","last_undegraded":"2024-09-21T02:03:59.781695+0000","last_fullsized":"2024-09-21T02:03:59.781695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'10","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:23.673341+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:04:23.673341+0000","last_peered":"2024-09-21T02:04:23.673341+0000","last_clean":"2024-09-21T02:04:23.673341+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:04:23.673341+0000","last_undegraded":"2024-09-21T02:04:23.673341+0000","last_fullsized":"2024-09-21T02:04:23.673341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075568+0000","last_change":"2024-09-21T02:03:58.080947+0000","last_active":"2024-09-21T02:03:59.075568+0000","last_peered":"2024-09-21T02:03:59.075568+0000","last_clean":"2024-09-21T02:03:59.075568+0000","last_became_active":"2024-09-21T02:03:58.080632+0000","last_became_peered":"2024-09-21T02:03:58.080632+0000","last_unstale":"2024-09-21T02:03:59.075568+0000","last_undegraded":"2024-09-21T02:03:59.075568+0000","last_fullsized":"2024-09-21T02:03:59.075568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:40.307689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140004+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:15.140004+0000","last_peered":"2024-09-21T02:04:15.140004+0000","last_clean":"2024-09-21T02:04:15.140004+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:15.140004+0000","last_undegraded":"2024-09-21T02:04:15.140004+0000","last_fullsized":"2024-09-21T02:04:15.140004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836343+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:03:59.836343+0000","last_peered":"2024-09-21T02:03:59.836343+0000","last_clean":"2024-09-21T02:03:59.836343+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:03:59.836343+0000","last_undegraded":"2024-09-21T02:03:59.836343+0000","last_fullsized":"2024-09-21T02:03:59.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.109035+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:04:08.109035+0000","last_peered":"2024-09-21T02:04:08.109035+0000","last_clean":"2024-09-21T02:04:08.109035+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:04:08.109035+0000","last_undegraded":"2024-09-21T02:04:08.109035+0000","last_fullsized":"2024-09-21T02:04:08.109035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075210+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:03:59.075210+0000","last_peered":"2024-09-21T02:03:59.075210+0000","last_clean":"2024-09-21T02:03:59.075210+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:03:59.075210+0000","last_undegraded":"2024-09-21T02:03:59.075210+0000","last_fullsized":"2024-09-21T02:03:59.075210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.141799+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:04:15.141799+0000","last_peered":"2024-09-21T02:04:15.141799+0000","last_clean":"2024-09-21T02:04:15.141799+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:04:15.141799+0000","last_undegraded":"2024-09-21T02:04:15.141799+0000","last_fullsized":"2024-09-21T02:04:15.141799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835641+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:03:59.835641+0000","last_peered":"2024-09-21T02:03:59.835641+0000","last_clean":"2024-09-21T02:03:59.835641+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:03:59.835641+0000","last_undegraded":"2024-09-21T02:03:59.835641+0000","last_fullsized":"2024-09-21T02:03:59.835641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":30,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.144449+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:04:15.144449+0000","last_peered":"2024-09-21T02:04:15.144449+0000","last_clean":"2024-09-21T02:04:15.144449+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:04:15.144449+0000","last_undegraded":"2024-09-21T02:04:15.144449+0000","last_fullsized":"2024-09-21T02:04:15.144449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074582+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:03:59.074582+0000","last_peered":"2024-09-21T02:03:59.074582+0000","last_clean":"2024-09-21T02:03:59.074582+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:03:59.074582+0000","last_undegraded":"2024-09-21T02:03:59.074582+0000","last_fullsized":"2024-09-21T02:03:59.074582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069544+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:04:03.069544+0000","last_peered":"2024-09-21T02:04:03.069544+0000","last_clean":"2024-09-21T02:04:03.069544+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:04:03.069544+0000","last_undegraded":"2024-09-21T02:04:03.069544+0000","last_fullsized":"2024-09-21T02:04:03.069544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074555+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:03:59.074555+0000","last_peered":"2024-09-21T02:03:59.074555+0000","last_clean":"2024-09-21T02:03:59.074555+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:03:59.074555+0000","last_undegraded":"2024-09-21T02:03:59.074555+0000","last_fullsized":"2024-09-21T02:03:59.074555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.071532+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:03.071532+0000","last_peered":"2024-09-21T02:04:03.071532+0000","last_clean":"2024-09-21T02:04:03.071532+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:03.071532+0000","last_undegraded":"2024-09-21T02:04:03.071532+0000","last_fullsized":"2024-09-21T02:04:03.071532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074911+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:03:59.074911+0000","last_peered":"2024-09-21T02:03:59.074911+0000","last_clean":"2024-09-21T02:03:59.074911+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:03:59.074911+0000","last_undegraded":"2024-09-21T02:03:59.074911+0000","last_fullsized":"2024-09-21T02:03:59.074911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073109+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:03:59.073109+0000","last_peered":"2024-09-21T02:03:59.073109+0000","last_clean":"2024-09-21T02:03:59.073109+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:03:59.073109+0000","last_undegraded":"2024-09-21T02:03:59.073109+0000","last_fullsized":"2024-09-21T02:03:59.073109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075681+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:03:59.075681+0000","last_peered":"2024-09-21T02:03:59.075681+0000","last_clean":"2024-09-21T02:03:59.075681+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:03:59.075681+0000","last_undegraded":"2024-09-21T02:03:59.075681+0000","last_fullsized":"2024-09-21T02:03:59.075681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140381+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:04:15.140381+0000","last_peered":"2024-09-21T02:04:15.140381+0000","last_clean":"2024-09-21T02:04:15.140381+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:04:15.140381+0000","last_undegraded":"2024-09-21T02:04:15.140381+0000","last_fullsized":"2024-09-21T02:04:15.140381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075817+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:03:59.075817+0000","last_peered":"2024-09-21T02:03:59.075817+0000","last_clean":"2024-09-21T02:03:59.075817+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:03:59.075817+0000","last_undegraded":"2024-09-21T02:03:59.075817+0000","last_fullsized":"2024-09-21T02:03:59.075817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074041+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:03:59.074041+0000","last_peered":"2024-09-21T02:03:59.074041+0000","last_clean":"2024-09-21T02:03:59.074041+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:03:59.074041+0000","last_undegraded":"2024-09-21T02:03:59.074041+0000","last_fullsized":"2024-09-21T02:03:59.074041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.109669+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:04:08.109669+0000","last_peered":"2024-09-21T02:04:08.109669+0000","last_clean":"2024-09-21T02:04:08.109669+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:04:08.109669+0000","last_undegraded":"2024-09-21T02:04:08.109669+0000","last_fullsized":"2024-09-21T02:04:08.109669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835923+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:03:59.835923+0000","last_peered":"2024-09-21T02:03:59.835923+0000","last_clean":"2024-09-21T02:03:59.835923+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:03:59.835923+0000","last_undegraded":"2024-09-21T02:03:59.835923+0000","last_fullsized":"2024-09-21T02:03:59.835923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.144040+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:04:15.144040+0000","last_peered":"2024-09-21T02:04:15.144040+0000","last_clean":"2024-09-21T02:04:15.144040+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:04:15.144040+0000","last_undegraded":"2024-09-21T02:04:15.144040+0000","last_fullsized":"2024-09-21T02:04:15.144040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835986+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:03:59.835986+0000","last_peered":"2024-09-21T02:03:59.835986+0000","last_clean":"2024-09-21T02:03:59.835986+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:03:59.835986+0000","last_undegraded":"2024-09-21T02:03:59.835986+0000","last_fullsized":"2024-09-21T02:03:59.835986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:05.466556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140513+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:04:15.140513+0000","last_peered":"2024-09-21T02:04:15.140513+0000","last_clean":"2024-09-21T02:04:15.140513+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:04:15.140513+0000","last_undegraded":"2024-09-21T02:04:15.140513+0000","last_fullsized":"2024-09-21T02:04:15.140513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072781+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:03:59.072781+0000","last_peered":"2024-09-21T02:03:59.072781+0000","last_clean":"2024-09-21T02:03:59.072781+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:03:59.072781+0000","last_undegraded":"2024-09-21T02:03:59.072781+0000","last_fullsized":"2024-09-21T02:03:59.072781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075545+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:03:59.075545+0000","last_peered":"2024-09-21T02:03:59.075545+0000","last_clean":"2024-09-21T02:03:59.075545+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:03:59.075545+0000","last_undegraded":"2024-09-21T02:03:59.075545+0000","last_fullsized":"2024-09-21T02:03:59.075545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781505+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:03:59.781505+0000","last_peered":"2024-09-21T02:03:59.781505+0000","last_clean":"2024-09-21T02:03:59.781505+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:03:59.781505+0000","last_undegraded":"2024-09-21T02:03:59.781505+0000","last_fullsized":"2024-09-21T02:03:59.781505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072896+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:03:59.072896+0000","last_peered":"2024-09-21T02:03:59.072896+0000","last_clean":"2024-09-21T02:03:59.072896+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:03:59.072896+0000","last_undegraded":"2024-09-21T02:03:59.072896+0000","last_fullsized":"2024-09-21T02:03:59.072896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076487+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:03:59.076487+0000","last_peered":"2024-09-21T02:03:59.076487+0000","last_clean":"2024-09-21T02:03:59.076487+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:03:59.076487+0000","last_undegraded":"2024-09-21T02:03:59.076487+0000","last_fullsized":"2024-09-21T02:03:59.076487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075996+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:03:59.075996+0000","last_peered":"2024-09-21T02:03:59.075996+0000","last_clean":"2024-09-21T02:03:59.075996+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:03:59.075996+0000","last_undegraded":"2024-09-21T02:03:59.075996+0000","last_fullsized":"2024-09-21T02:03:59.075996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073021+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:03:59.073021+0000","last_peered":"2024-09-21T02:03:59.073021+0000","last_clean":"2024-09-21T02:03:59.073021+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:03:59.073021+0000","last_undegraded":"2024-09-21T02:03:59.073021+0000","last_fullsized":"2024-09-21T02:03:59.073021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.139792+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:04:15.139792+0000","last_peered":"2024-09-21T02:04:15.139792+0000","last_clean":"2024-09-21T02:04:15.139792+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:04:15.139792+0000","last_undegraded":"2024-09-21T02:04:15.139792+0000","last_fullsized":"2024-09-21T02:04:15.139792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836949+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:03:59.836949+0000","last_peered":"2024-09-21T02:03:59.836949+0000","last_clean":"2024-09-21T02:03:59.836949+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:03:59.836949+0000","last_undegraded":"2024-09-21T02:03:59.836949+0000","last_fullsized":"2024-09-21T02:03:59.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107333+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:04:08.107333+0000","last_peered":"2024-09-21T02:04:08.107333+0000","last_clean":"2024-09-21T02:04:08.107333+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:04:08.107333+0000","last_undegraded":"2024-09-21T02:04:08.107333+0000","last_fullsized":"2024-09-21T02:04:08.107333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836206+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:03:59.836206+0000","last_peered":"2024-09-21T02:03:59.836206+0000","last_clean":"2024-09-21T02:03:59.836206+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:03:59.836206+0000","last_undegraded":"2024-09-21T02:03:59.836206+0000","last_fullsized":"2024-09-21T02:03:59.836206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.103523+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:04:08.103523+0000","last_peered":"2024-09-21T02:04:08.103523+0000","last_clean":"2024-09-21T02:04:08.103523+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:04:08.103523+0000","last_undegraded":"2024-09-21T02:04:08.103523+0000","last_fullsized":"2024-09-21T02:04:08.103523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781583+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:03:59.781583+0000","last_peered":"2024-09-21T02:03:59.781583+0000","last_clean":"2024-09-21T02:03:59.781583+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:03:59.781583+0000","last_undegraded":"2024-09-21T02:03:59.781583+0000","last_fullsized":"2024-09-21T02:03:59.781583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107012+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:04:08.107012+0000","last_peered":"2024-09-21T02:04:08.107012+0000","last_clean":"2024-09-21T02:04:08.107012+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:04:08.107012+0000","last_undegraded":"2024-09-21T02:04:08.107012+0000","last_fullsized":"2024-09-21T02:04:08.107012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835669+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:03:59.835669+0000","last_peered":"2024-09-21T02:03:59.835669+0000","last_clean":"2024-09-21T02:03:59.835669+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:03:59.835669+0000","last_undegraded":"2024-09-21T02:03:59.835669+0000","last_fullsized":"2024-09-21T02:03:59.835669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072911+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:03:59.072911+0000","last_peered":"2024-09-21T02:03:59.072911+0000","last_clean":"2024-09-21T02:03:59.072911+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:03:59.072911+0000","last_undegraded":"2024-09-21T02:03:59.072911+0000","last_fullsized":"2024-09-21T02:03:59.072911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836615+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:03:59.836615+0000","last_peered":"2024-09-21T02:03:59.836615+0000","last_clean":"2024-09-21T02:03:59.836615+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:03:59.836615+0000","last_undegraded":"2024-09-21T02:03:59.836615+0000","last_fullsized":"2024-09-21T02:03:59.836615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.104423+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:04:08.104423+0000","last_peered":"2024-09-21T02:04:08.104423+0000","last_clean":"2024-09-21T02:04:08.104423+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:04:08.104423+0000","last_undegraded":"2024-09-21T02:04:08.104423+0000","last_fullsized":"2024-09-21T02:04:08.104423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835473+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:03:59.835473+0000","last_peered":"2024-09-21T02:03:59.835473+0000","last_clean":"2024-09-21T02:03:59.835473+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:03:59.835473+0000","last_undegraded":"2024-09-21T02:03:59.835473+0000","last_fullsized":"2024-09-21T02:03:59.835473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076421+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:03:59.076421+0000","last_peered":"2024-09-21T02:03:59.076421+0000","last_clean":"2024-09-21T02:03:59.076421+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:03:59.076421+0000","last_undegraded":"2024-09-21T02:03:59.076421+0000","last_fullsized":"2024-09-21T02:03:59.076421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069209+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:03.069209+0000","last_peered":"2024-09-21T02:04:03.069209+0000","last_clean":"2024-09-21T02:04:03.069209+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:03.069209+0000","last_undegraded":"2024-09-21T02:04:03.069209+0000","last_fullsized":"2024-09-21T02:04:03.069209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075820+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:03:59.075820+0000","last_peered":"2024-09-21T02:03:59.075820+0000","last_clean":"2024-09-21T02:03:59.075820+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:03:59.075820+0000","last_undegraded":"2024-09-21T02:03:59.075820+0000","last_fullsized":"2024-09-21T02:03:59.075820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.137573+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:04:15.137573+0000","last_peered":"2024-09-21T02:04:15.137573+0000","last_clean":"2024-09-21T02:04:15.137573+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:04:15.137573+0000","last_undegraded":"2024-09-21T02:04:15.137573+0000","last_fullsized":"2024-09-21T02:04:15.137573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836754+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:03:59.836754+0000","last_peered":"2024-09-21T02:03:59.836754+0000","last_clean":"2024-09-21T02:03:59.836754+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:03:59.836754+0000","last_undegraded":"2024-09-21T02:03:59.836754+0000","last_fullsized":"2024-09-21T02:03:59.836754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069853+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:04:03.069853+0000","last_peered":"2024-09-21T02:04:03.069853+0000","last_clean":"2024-09-21T02:04:03.069853+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:04:03.069853+0000","last_undegraded":"2024-09-21T02:04:03.069853+0000","last_fullsized":"2024-09-21T02:04:03.069853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837178+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:03:59.837178+0000","last_peered":"2024-09-21T02:03:59.837178+0000","last_clean":"2024-09-21T02:03:59.837178+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:03:59.837178+0000","last_undegraded":"2024-09-21T02:03:59.837178+0000","last_fullsized":"2024-09-21T02:03:59.837178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140116+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:04:15.140116+0000","last_peered":"2024-09-21T02:04:15.140116+0000","last_clean":"2024-09-21T02:04:15.140116+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:04:15.140116+0000","last_undegraded":"2024-09-21T02:04:15.140116+0000","last_fullsized":"2024-09-21T02:04:15.140116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5734,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":823296,"data_stored":664431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":74125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":73717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":75327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":73565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":73691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35568,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336272,"statfs":{"total":96636764160,"available":96600342528,"internally_reserved":0,"allocated":229376,"data_stored":75549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":663775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:26.993 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:04:27.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 -- 172.21.15.88:0/3773952286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c8bd0 msgr2=0x7f05780cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 --2- 172.21.15.88:0/3773952286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c8bd0 0x7f05780cbc80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0574009830 tx=0x7f057402a480 comp rx=0 tx=0).stop 2024-09-21T02:04:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 -- 172.21.15.88:0/3773952286 shutdown_connections 2024-09-21T02:04:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 --2- 172.21.15.88:0/3773952286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05780c4df0 0x7f05780c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 --2- 172.21.15.88:0/3773952286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c8bd0 0x7f05780cbc80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 -- 172.21.15.88:0/3773952286 >> 172.21.15.88:0/3773952286 conn(0x7f05780694f0 msgr2=0x7f0578072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 -- 172.21.15.88:0/3773952286 shutdown_connections 2024-09-21T02:04:27.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.140+0000 7f0580139640 1 -- 172.21.15.88:0/3773952286 wait complete. 2024-09-21T02:04:27.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.142+0000 7f0580139640 1 Processor -- start 2024-09-21T02:04:27.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.142+0000 7f0580139640 1 -- start start 2024-09-21T02:04:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f0580139640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c4df0 0x7f057815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f0580139640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05780c5200 0x7f0578151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f0580139640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05780c8bd0 0x7f0578152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f0580139640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05780ca1c0 con 0x7f05780c4df0 2024-09-21T02:04:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f0580139640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f05780c9f20 con 0x7f05780c8bd0 2024-09-21T02:04:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f0580139640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f05780ca4a0 con 0x7f05780c5200 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c4df0 0x7f057815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c4df0 0x7f057815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49780/0 (socket says 172.21.15.88:49780) 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 -- 172.21.15.88:0/2028187336 learned_addr learned my addr 172.21.15.88:0/2028187336 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057d6d4640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05780c5200 0x7f0578151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057e6d6640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05780c8bd0 0x7f0578152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 -- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05780c5200 msgr2=0x7f0578151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05780c5200 0x7f0578151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 -- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05780c8bd0 msgr2=0x7f0578152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05780c8bd0 0x7f0578152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057ded5640 1 -- 172.21.15.88:0/2028187336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057815dbc0 con 0x7f05780c4df0 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.143+0000 7f057d6d4640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05780c5200 0x7f0578151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.144+0000 7f057e6d6640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05780c8bd0 0x7f0578152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.144+0000 7f057ded5640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c4df0 0x7f057815ced0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0574009950 tx=0x7f0574033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.144+0000 7f056e7fc640 1 -- 172.21.15.88:0/2028187336 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574031d60 con 0x7f05780c4df0 2024-09-21T02:04:27.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.144+0000 7f056e7fc640 1 -- 172.21.15.88:0/2028187336 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0574033e10 con 0x7f05780c4df0 2024-09-21T02:04:27.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.144+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05780cfbf0 con 0x7f05780c4df0 2024-09-21T02:04:27.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.144+0000 7f056e7fc640 1 -- 172.21.15.88:0/2028187336 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574005260 con 0x7f05780c4df0 2024-09-21T02:04:27.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.144+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05781e41c0 con 0x7f05780c4df0 2024-09-21T02:04:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.145+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05780cc230 con 0x7f05780c4df0 2024-09-21T02:04:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.146+0000 7f056e7fc640 1 -- 172.21.15.88:0/2028187336 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0574005400 con 0x7f05780c4df0 2024-09-21T02:04:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.146+0000 7f056e7fc640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f055c078bc0 0x7f055c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.146+0000 7f056e7fc640 1 -- 172.21.15.88:0/2028187336 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f05740bc080 con 0x7f05780c4df0 2024-09-21T02:04:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.146+0000 7f057d6d4640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f055c078bc0 0x7f055c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.147+0000 7f057d6d4640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f055c078bc0 0x7f055c07b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05780cbcf0 tx=0x7f0568005cb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.151+0000 7f056e7fc640 1 -- 172.21.15.88:0/2028187336 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f057403f030 con 0x7f05780c4df0 2024-09-21T02:04:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.327+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05780cfeb0 con 0x7f055c078bc0 2024-09-21T02:04:27.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.332+0000 7f056e7fc640 1 -- 172.21.15.88:0/2028187336 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251884 (secure 0 0 0) 0x7f05780cfeb0 con 0x7f055c078bc0 2024-09-21T02:04:27.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:27.337 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:04:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f055c078bc0 msgr2=0x7f055c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f055c078bc0 0x7f055c07b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05780cbcf0 tx=0x7f0568005cb0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c4df0 msgr2=0x7f057815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c4df0 0x7f057815ced0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0574009950 tx=0x7f0574033c20 comp rx=0 tx=0).stop 2024-09-21T02:04:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 shutdown_connections 2024-09-21T02:04:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f055c078bc0 0x7f055c07b0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05780c8bd0 0x7f0578152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05780c5200 0x7f0578151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 --2- 172.21.15.88:0/2028187336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05780c4df0 0x7f057815ced0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 >> 172.21.15.88:0/2028187336 conn(0x7f05780694f0 msgr2=0x7f0578157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 shutdown_connections 2024-09-21T02:04:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.335+0000 7f0580139640 1 -- 172.21.15.88:0/2028187336 wait complete. 2024-09-21T02:04:27.348 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-21T02:04:26.824932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596102,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":229,"num_write_kb":1669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239944,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734776,"statfs":{"total":773094113280,"available":772848410624,"internally_reserved":0,"allocated":3022848,"data_stored":1774180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1918,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074727+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:03:59.074727+0000","last_peered":"2024-09-21T02:03:59.074727+0000","last_clean":"2024-09-21T02:03:59.074727+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:03:59.074727+0000","last_undegraded":"2024-09-21T02:03:59.074727+0000","last_fullsized":"2024-09-21T02:03:59.074727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073213+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:03:59.073213+0000","last_peered":"2024-09-21T02:03:59.073213+0000","last_clean":"2024-09-21T02:03:59.073213+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:03:59.073213+0000","last_undegraded":"2024-09-21T02:03:59.073213+0000","last_fullsized":"2024-09-21T02:03:59.073213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836539+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:03:59.836539+0000","last_peered":"2024-09-21T02:03:59.836539+0000","last_clean":"2024-09-21T02:03:59.836539+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:03:59.836539+0000","last_undegraded":"2024-09-21T02:03:59.836539+0000","last_fullsized":"2024-09-21T02:03:59.836539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075220+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:03:59.075220+0000","last_peered":"2024-09-21T02:03:59.075220+0000","last_clean":"2024-09-21T02:03:59.075220+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:03:59.075220+0000","last_undegraded":"2024-09-21T02:03:59.075220+0000","last_fullsized":"2024-09-21T02:03:59.075220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074990+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:03:59.074990+0000","last_peered":"2024-09-21T02:03:59.074990+0000","last_clean":"2024-09-21T02:03:59.074990+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:03:59.074990+0000","last_undegraded":"2024-09-21T02:03:59.074990+0000","last_fullsized":"2024-09-21T02:03:59.074990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074223+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:03:59.074223+0000","last_peered":"2024-09-21T02:03:59.074223+0000","last_clean":"2024-09-21T02:03:59.074223+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:03:59.074223+0000","last_undegraded":"2024-09-21T02:03:59.074223+0000","last_fullsized":"2024-09-21T02:03:59.074223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074095+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:03:59.074095+0000","last_peered":"2024-09-21T02:03:59.074095+0000","last_clean":"2024-09-21T02:03:59.074095+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:03:59.074095+0000","last_undegraded":"2024-09-21T02:03:59.074095+0000","last_fullsized":"2024-09-21T02:03:59.074095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835347+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:03:59.835347+0000","last_peered":"2024-09-21T02:03:59.835347+0000","last_clean":"2024-09-21T02:03:59.835347+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:03:59.835347+0000","last_undegraded":"2024-09-21T02:03:59.835347+0000","last_fullsized":"2024-09-21T02:03:59.835347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074637+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:03:59.074637+0000","last_peered":"2024-09-21T02:03:59.074637+0000","last_clean":"2024-09-21T02:03:59.074637+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:03:59.074637+0000","last_undegraded":"2024-09-21T02:03:59.074637+0000","last_fullsized":"2024-09-21T02:03:59.074637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835966+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:03:59.835966+0000","last_peered":"2024-09-21T02:03:59.835966+0000","last_clean":"2024-09-21T02:03:59.835966+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:03:59.835966+0000","last_undegraded":"2024-09-21T02:03:59.835966+0000","last_fullsized":"2024-09-21T02:03:59.835966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835587+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:03:59.835587+0000","last_peered":"2024-09-21T02:03:59.835587+0000","last_clean":"2024-09-21T02:03:59.835587+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:03:59.835587+0000","last_undegraded":"2024-09-21T02:03:59.835587+0000","last_fullsized":"2024-09-21T02:03:59.835587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781637+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:03:59.781637+0000","last_peered":"2024-09-21T02:03:59.781637+0000","last_clean":"2024-09-21T02:03:59.781637+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:03:59.781637+0000","last_undegraded":"2024-09-21T02:03:59.781637+0000","last_fullsized":"2024-09-21T02:03:59.781637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835918+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:03:59.835918+0000","last_peered":"2024-09-21T02:03:59.835918+0000","last_clean":"2024-09-21T02:03:59.835918+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:03:59.835918+0000","last_undegraded":"2024-09-21T02:03:59.835918+0000","last_fullsized":"2024-09-21T02:03:59.835918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836186+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:03:59.836186+0000","last_peered":"2024-09-21T02:03:59.836186+0000","last_clean":"2024-09-21T02:03:59.836186+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:03:59.836186+0000","last_undegraded":"2024-09-21T02:03:59.836186+0000","last_fullsized":"2024-09-21T02:03:59.836186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072763+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:03:59.072763+0000","last_peered":"2024-09-21T02:03:59.072763+0000","last_clean":"2024-09-21T02:03:59.072763+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:03:59.072763+0000","last_undegraded":"2024-09-21T02:03:59.072763+0000","last_fullsized":"2024-09-21T02:03:59.072763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075548+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:03:59.075548+0000","last_peered":"2024-09-21T02:03:59.075548+0000","last_clean":"2024-09-21T02:03:59.075548+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:03:59.075548+0000","last_undegraded":"2024-09-21T02:03:59.075548+0000","last_fullsized":"2024-09-21T02:03:59.075548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075910+0000","last_change":"2024-09-21T02:03:58.074570+0000","last_active":"2024-09-21T02:03:59.075910+0000","last_peered":"2024-09-21T02:03:59.075910+0000","last_clean":"2024-09-21T02:03:59.075910+0000","last_became_active":"2024-09-21T02:03:58.074271+0000","last_became_peered":"2024-09-21T02:03:58.074271+0000","last_unstale":"2024-09-21T02:03:59.075910+0000","last_undegraded":"2024-09-21T02:03:59.075910+0000","last_fullsized":"2024-09-21T02:03:59.075910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.292792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835423+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:03:59.835423+0000","last_peered":"2024-09-21T02:03:59.835423+0000","last_clean":"2024-09-21T02:03:59.835423+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:03:59.835423+0000","last_undegraded":"2024-09-21T02:03:59.835423+0000","last_fullsized":"2024-09-21T02:03:59.835423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076132+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:03:59.076132+0000","last_peered":"2024-09-21T02:03:59.076132+0000","last_clean":"2024-09-21T02:03:59.076132+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:03:59.076132+0000","last_undegraded":"2024-09-21T02:03:59.076132+0000","last_fullsized":"2024-09-21T02:03:59.076132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835510+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:03:59.835510+0000","last_peered":"2024-09-21T02:03:59.835510+0000","last_clean":"2024-09-21T02:03:59.835510+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:03:59.835510+0000","last_undegraded":"2024-09-21T02:03:59.835510+0000","last_fullsized":"2024-09-21T02:03:59.835510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836526+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:03:59.836526+0000","last_peered":"2024-09-21T02:03:59.836526+0000","last_clean":"2024-09-21T02:03:59.836526+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:03:59.836526+0000","last_undegraded":"2024-09-21T02:03:59.836526+0000","last_fullsized":"2024-09-21T02:03:59.836526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075251+0000","last_change":"2024-09-21T02:03:58.076378+0000","last_active":"2024-09-21T02:03:59.075251+0000","last_peered":"2024-09-21T02:03:59.075251+0000","last_clean":"2024-09-21T02:03:59.075251+0000","last_became_active":"2024-09-21T02:03:58.076075+0000","last_became_peered":"2024-09-21T02:03:58.076075+0000","last_unstale":"2024-09-21T02:03:59.075251+0000","last_undegraded":"2024-09-21T02:03:59.075251+0000","last_fullsized":"2024-09-21T02:03:59.075251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:54.775213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835596+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:03:59.835596+0000","last_peered":"2024-09-21T02:03:59.835596+0000","last_clean":"2024-09-21T02:03:59.835596+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:03:59.835596+0000","last_undegraded":"2024-09-21T02:03:59.835596+0000","last_fullsized":"2024-09-21T02:03:59.835596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074571+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:03:59.074571+0000","last_peered":"2024-09-21T02:03:59.074571+0000","last_clean":"2024-09-21T02:03:59.074571+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:03:59.074571+0000","last_undegraded":"2024-09-21T02:03:59.074571+0000","last_fullsized":"2024-09-21T02:03:59.074571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836762+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:03:59.836762+0000","last_peered":"2024-09-21T02:03:59.836762+0000","last_clean":"2024-09-21T02:03:59.836762+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:03:59.836762+0000","last_undegraded":"2024-09-21T02:03:59.836762+0000","last_fullsized":"2024-09-21T02:03:59.836762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835955+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:03:59.835955+0000","last_peered":"2024-09-21T02:03:59.835955+0000","last_clean":"2024-09-21T02:03:59.835955+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:03:59.835955+0000","last_undegraded":"2024-09-21T02:03:59.835955+0000","last_fullsized":"2024-09-21T02:03:59.835955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073053+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:03:59.073053+0000","last_peered":"2024-09-21T02:03:59.073053+0000","last_clean":"2024-09-21T02:03:59.073053+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:03:59.073053+0000","last_undegraded":"2024-09-21T02:03:59.073053+0000","last_fullsized":"2024-09-21T02:03:59.073053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836383+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:03:59.836383+0000","last_peered":"2024-09-21T02:03:59.836383+0000","last_clean":"2024-09-21T02:03:59.836383+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:03:59.836383+0000","last_undegraded":"2024-09-21T02:03:59.836383+0000","last_fullsized":"2024-09-21T02:03:59.836383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836861+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:03:59.836861+0000","last_peered":"2024-09-21T02:03:59.836861+0000","last_clean":"2024-09-21T02:03:59.836861+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:03:59.836861+0000","last_undegraded":"2024-09-21T02:03:59.836861+0000","last_fullsized":"2024-09-21T02:03:59.836861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075554+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:03:59.075554+0000","last_peered":"2024-09-21T02:03:59.075554+0000","last_clean":"2024-09-21T02:03:59.075554+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:03:59.075554+0000","last_undegraded":"2024-09-21T02:03:59.075554+0000","last_fullsized":"2024-09-21T02:03:59.075554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835391+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:03:59.835391+0000","last_peered":"2024-09-21T02:03:59.835391+0000","last_clean":"2024-09-21T02:03:59.835391+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:03:59.835391+0000","last_undegraded":"2024-09-21T02:03:59.835391+0000","last_fullsized":"2024-09-21T02:03:59.835391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835439+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:03:59.835439+0000","last_peered":"2024-09-21T02:03:59.835439+0000","last_clean":"2024-09-21T02:03:59.835439+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:03:59.835439+0000","last_undegraded":"2024-09-21T02:03:59.835439+0000","last_fullsized":"2024-09-21T02:03:59.835439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781525+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:03:59.781525+0000","last_peered":"2024-09-21T02:03:59.781525+0000","last_clean":"2024-09-21T02:03:59.781525+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:03:59.781525+0000","last_undegraded":"2024-09-21T02:03:59.781525+0000","last_fullsized":"2024-09-21T02:03:59.781525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107531+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:04:08.107531+0000","last_peered":"2024-09-21T02:04:08.107531+0000","last_clean":"2024-09-21T02:04:08.107531+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:04:08.107531+0000","last_undegraded":"2024-09-21T02:04:08.107531+0000","last_fullsized":"2024-09-21T02:04:08.107531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836856+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:03:59.836856+0000","last_peered":"2024-09-21T02:03:59.836856+0000","last_clean":"2024-09-21T02:03:59.836856+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:03:59.836856+0000","last_undegraded":"2024-09-21T02:03:59.836856+0000","last_fullsized":"2024-09-21T02:03:59.836856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.137774+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:04:15.137774+0000","last_peered":"2024-09-21T02:04:15.137774+0000","last_clean":"2024-09-21T02:04:15.137774+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:04:15.137774+0000","last_undegraded":"2024-09-21T02:04:15.137774+0000","last_fullsized":"2024-09-21T02:04:15.137774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.137911+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:04:15.137911+0000","last_peered":"2024-09-21T02:04:15.137911+0000","last_clean":"2024-09-21T02:04:15.137911+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:04:15.137911+0000","last_undegraded":"2024-09-21T02:04:15.137911+0000","last_fullsized":"2024-09-21T02:04:15.137911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835573+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:03:59.835573+0000","last_peered":"2024-09-21T02:03:59.835573+0000","last_clean":"2024-09-21T02:03:59.835573+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:03:59.835573+0000","last_undegraded":"2024-09-21T02:03:59.835573+0000","last_fullsized":"2024-09-21T02:03:59.835573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074720+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:03:59.074720+0000","last_peered":"2024-09-21T02:03:59.074720+0000","last_clean":"2024-09-21T02:03:59.074720+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:03:59.074720+0000","last_undegraded":"2024-09-21T02:03:59.074720+0000","last_fullsized":"2024-09-21T02:03:59.074720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072701+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:03:59.072701+0000","last_peered":"2024-09-21T02:03:59.072701+0000","last_clean":"2024-09-21T02:03:59.072701+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:03:59.072701+0000","last_undegraded":"2024-09-21T02:03:59.072701+0000","last_fullsized":"2024-09-21T02:03:59.072701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.108080+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:04:08.108080+0000","last_peered":"2024-09-21T02:04:08.108080+0000","last_clean":"2024-09-21T02:04:08.108080+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:04:08.108080+0000","last_undegraded":"2024-09-21T02:04:08.108080+0000","last_fullsized":"2024-09-21T02:04:08.108080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074748+0000","last_change":"2024-09-21T02:03:58.075434+0000","last_active":"2024-09-21T02:03:59.074748+0000","last_peered":"2024-09-21T02:03:59.074748+0000","last_clean":"2024-09-21T02:03:59.074748+0000","last_became_active":"2024-09-21T02:03:58.075130+0000","last_became_peered":"2024-09-21T02:03:58.075130+0000","last_unstale":"2024-09-21T02:03:59.074748+0000","last_undegraded":"2024-09-21T02:03:59.074748+0000","last_fullsized":"2024-09-21T02:03:59.074748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:56.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107586+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:04:08.107586+0000","last_peered":"2024-09-21T02:04:08.107586+0000","last_clean":"2024-09-21T02:04:08.107586+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:04:08.107586+0000","last_undegraded":"2024-09-21T02:04:08.107586+0000","last_fullsized":"2024-09-21T02:04:08.107586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836482+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:03:59.836482+0000","last_peered":"2024-09-21T02:03:59.836482+0000","last_clean":"2024-09-21T02:03:59.836482+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:03:59.836482+0000","last_undegraded":"2024-09-21T02:03:59.836482+0000","last_fullsized":"2024-09-21T02:03:59.836482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.070253+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:04:03.070253+0000","last_peered":"2024-09-21T02:04:03.070253+0000","last_clean":"2024-09-21T02:04:03.070253+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:04:03.070253+0000","last_undegraded":"2024-09-21T02:04:03.070253+0000","last_fullsized":"2024-09-21T02:04:03.070253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781695+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:03:59.781695+0000","last_peered":"2024-09-21T02:03:59.781695+0000","last_clean":"2024-09-21T02:03:59.781695+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:03:59.781695+0000","last_undegraded":"2024-09-21T02:03:59.781695+0000","last_fullsized":"2024-09-21T02:03:59.781695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'10","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:23.673341+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:04:23.673341+0000","last_peered":"2024-09-21T02:04:23.673341+0000","last_clean":"2024-09-21T02:04:23.673341+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:04:23.673341+0000","last_undegraded":"2024-09-21T02:04:23.673341+0000","last_fullsized":"2024-09-21T02:04:23.673341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075568+0000","last_change":"2024-09-21T02:03:58.080947+0000","last_active":"2024-09-21T02:03:59.075568+0000","last_peered":"2024-09-21T02:03:59.075568+0000","last_clean":"2024-09-21T02:03:59.075568+0000","last_became_active":"2024-09-21T02:03:58.080632+0000","last_became_peered":"2024-09-21T02:03:58.080632+0000","last_unstale":"2024-09-21T02:03:59.075568+0000","last_undegraded":"2024-09-21T02:03:59.075568+0000","last_fullsized":"2024-09-21T02:03:59.075568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:40.307689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140004+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:15.140004+0000","last_peered":"2024-09-21T02:04:15.140004+0000","last_clean":"2024-09-21T02:04:15.140004+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:15.140004+0000","last_undegraded":"2024-09-21T02:04:15.140004+0000","last_fullsized":"2024-09-21T02:04:15.140004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836343+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:03:59.836343+0000","last_peered":"2024-09-21T02:03:59.836343+0000","last_clean":"2024-09-21T02:03:59.836343+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:03:59.836343+0000","last_undegraded":"2024-09-21T02:03:59.836343+0000","last_fullsized":"2024-09-21T02:03:59.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.109035+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:04:08.109035+0000","last_peered":"2024-09-21T02:04:08.109035+0000","last_clean":"2024-09-21T02:04:08.109035+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:04:08.109035+0000","last_undegraded":"2024-09-21T02:04:08.109035+0000","last_fullsized":"2024-09-21T02:04:08.109035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075210+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:03:59.075210+0000","last_peered":"2024-09-21T02:03:59.075210+0000","last_clean":"2024-09-21T02:03:59.075210+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:03:59.075210+0000","last_undegraded":"2024-09-21T02:03:59.075210+0000","last_fullsized":"2024-09-21T02:03:59.075210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.141799+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:04:15.141799+0000","last_peered":"2024-09-21T02:04:15.141799+0000","last_clean":"2024-09-21T02:04:15.141799+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:04:15.141799+0000","last_undegraded":"2024-09-21T02:04:15.141799+0000","last_fullsized":"2024-09-21T02:04:15.141799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835641+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:03:59.835641+0000","last_peered":"2024-09-21T02:03:59.835641+0000","last_clean":"2024-09-21T02:03:59.835641+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:03:59.835641+0000","last_undegraded":"2024-09-21T02:03:59.835641+0000","last_fullsized":"2024-09-21T02:03:59.835641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":30,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.144449+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:04:15.144449+0000","last_peered":"2024-09-21T02:04:15.144449+0000","last_clean":"2024-09-21T02:04:15.144449+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:04:15.144449+0000","last_undegraded":"2024-09-21T02:04:15.144449+0000","last_fullsized":"2024-09-21T02:04:15.144449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074582+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:03:59.074582+0000","last_peered":"2024-09-21T02:03:59.074582+0000","last_clean":"2024-09-21T02:03:59.074582+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:03:59.074582+0000","last_undegraded":"2024-09-21T02:03:59.074582+0000","last_fullsized":"2024-09-21T02:03:59.074582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069544+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:04:03.069544+0000","last_peered":"2024-09-21T02:04:03.069544+0000","last_clean":"2024-09-21T02:04:03.069544+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:04:03.069544+0000","last_undegraded":"2024-09-21T02:04:03.069544+0000","last_fullsized":"2024-09-21T02:04:03.069544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074555+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:03:59.074555+0000","last_peered":"2024-09-21T02:03:59.074555+0000","last_clean":"2024-09-21T02:03:59.074555+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:03:59.074555+0000","last_undegraded":"2024-09-21T02:03:59.074555+0000","last_fullsized":"2024-09-21T02:03:59.074555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.071532+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:03.071532+0000","last_peered":"2024-09-21T02:04:03.071532+0000","last_clean":"2024-09-21T02:04:03.071532+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:03.071532+0000","last_undegraded":"2024-09-21T02:04:03.071532+0000","last_fullsized":"2024-09-21T02:04:03.071532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074911+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:03:59.074911+0000","last_peered":"2024-09-21T02:03:59.074911+0000","last_clean":"2024-09-21T02:03:59.074911+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:03:59.074911+0000","last_undegraded":"2024-09-21T02:03:59.074911+0000","last_fullsized":"2024-09-21T02:03:59.074911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073109+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:03:59.073109+0000","last_peered":"2024-09-21T02:03:59.073109+0000","last_clean":"2024-09-21T02:03:59.073109+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:03:59.073109+0000","last_undegraded":"2024-09-21T02:03:59.073109+0000","last_fullsized":"2024-09-21T02:03:59.073109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075681+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:03:59.075681+0000","last_peered":"2024-09-21T02:03:59.075681+0000","last_clean":"2024-09-21T02:03:59.075681+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:03:59.075681+0000","last_undegraded":"2024-09-21T02:03:59.075681+0000","last_fullsized":"2024-09-21T02:03:59.075681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140381+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:04:15.140381+0000","last_peered":"2024-09-21T02:04:15.140381+0000","last_clean":"2024-09-21T02:04:15.140381+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:04:15.140381+0000","last_undegraded":"2024-09-21T02:04:15.140381+0000","last_fullsized":"2024-09-21T02:04:15.140381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075817+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:03:59.075817+0000","last_peered":"2024-09-21T02:03:59.075817+0000","last_clean":"2024-09-21T02:03:59.075817+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:03:59.075817+0000","last_undegraded":"2024-09-21T02:03:59.075817+0000","last_fullsized":"2024-09-21T02:03:59.075817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.074041+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:03:59.074041+0000","last_peered":"2024-09-21T02:03:59.074041+0000","last_clean":"2024-09-21T02:03:59.074041+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:03:59.074041+0000","last_undegraded":"2024-09-21T02:03:59.074041+0000","last_fullsized":"2024-09-21T02:03:59.074041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.109669+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:04:08.109669+0000","last_peered":"2024-09-21T02:04:08.109669+0000","last_clean":"2024-09-21T02:04:08.109669+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:04:08.109669+0000","last_undegraded":"2024-09-21T02:04:08.109669+0000","last_fullsized":"2024-09-21T02:04:08.109669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835923+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:03:59.835923+0000","last_peered":"2024-09-21T02:03:59.835923+0000","last_clean":"2024-09-21T02:03:59.835923+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:03:59.835923+0000","last_undegraded":"2024-09-21T02:03:59.835923+0000","last_fullsized":"2024-09-21T02:03:59.835923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.144040+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:04:15.144040+0000","last_peered":"2024-09-21T02:04:15.144040+0000","last_clean":"2024-09-21T02:04:15.144040+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:04:15.144040+0000","last_undegraded":"2024-09-21T02:04:15.144040+0000","last_fullsized":"2024-09-21T02:04:15.144040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835986+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:03:59.835986+0000","last_peered":"2024-09-21T02:03:59.835986+0000","last_clean":"2024-09-21T02:03:59.835986+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:03:59.835986+0000","last_undegraded":"2024-09-21T02:03:59.835986+0000","last_fullsized":"2024-09-21T02:03:59.835986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:05.466556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140513+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:04:15.140513+0000","last_peered":"2024-09-21T02:04:15.140513+0000","last_clean":"2024-09-21T02:04:15.140513+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:04:15.140513+0000","last_undegraded":"2024-09-21T02:04:15.140513+0000","last_fullsized":"2024-09-21T02:04:15.140513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072781+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:03:59.072781+0000","last_peered":"2024-09-21T02:03:59.072781+0000","last_clean":"2024-09-21T02:03:59.072781+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:03:59.072781+0000","last_undegraded":"2024-09-21T02:03:59.072781+0000","last_fullsized":"2024-09-21T02:03:59.072781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075545+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:03:59.075545+0000","last_peered":"2024-09-21T02:03:59.075545+0000","last_clean":"2024-09-21T02:03:59.075545+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:03:59.075545+0000","last_undegraded":"2024-09-21T02:03:59.075545+0000","last_fullsized":"2024-09-21T02:03:59.075545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781505+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:03:59.781505+0000","last_peered":"2024-09-21T02:03:59.781505+0000","last_clean":"2024-09-21T02:03:59.781505+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:03:59.781505+0000","last_undegraded":"2024-09-21T02:03:59.781505+0000","last_fullsized":"2024-09-21T02:03:59.781505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072896+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:03:59.072896+0000","last_peered":"2024-09-21T02:03:59.072896+0000","last_clean":"2024-09-21T02:03:59.072896+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:03:59.072896+0000","last_undegraded":"2024-09-21T02:03:59.072896+0000","last_fullsized":"2024-09-21T02:03:59.072896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076487+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:03:59.076487+0000","last_peered":"2024-09-21T02:03:59.076487+0000","last_clean":"2024-09-21T02:03:59.076487+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:03:59.076487+0000","last_undegraded":"2024-09-21T02:03:59.076487+0000","last_fullsized":"2024-09-21T02:03:59.076487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075996+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:03:59.075996+0000","last_peered":"2024-09-21T02:03:59.075996+0000","last_clean":"2024-09-21T02:03:59.075996+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:03:59.075996+0000","last_undegraded":"2024-09-21T02:03:59.075996+0000","last_fullsized":"2024-09-21T02:03:59.075996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.073021+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:03:59.073021+0000","last_peered":"2024-09-21T02:03:59.073021+0000","last_clean":"2024-09-21T02:03:59.073021+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:03:59.073021+0000","last_undegraded":"2024-09-21T02:03:59.073021+0000","last_fullsized":"2024-09-21T02:03:59.073021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.139792+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:04:15.139792+0000","last_peered":"2024-09-21T02:04:15.139792+0000","last_clean":"2024-09-21T02:04:15.139792+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:04:15.139792+0000","last_undegraded":"2024-09-21T02:04:15.139792+0000","last_fullsized":"2024-09-21T02:04:15.139792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836949+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:03:59.836949+0000","last_peered":"2024-09-21T02:03:59.836949+0000","last_clean":"2024-09-21T02:03:59.836949+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:03:59.836949+0000","last_undegraded":"2024-09-21T02:03:59.836949+0000","last_fullsized":"2024-09-21T02:03:59.836949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107333+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:04:08.107333+0000","last_peered":"2024-09-21T02:04:08.107333+0000","last_clean":"2024-09-21T02:04:08.107333+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:04:08.107333+0000","last_undegraded":"2024-09-21T02:04:08.107333+0000","last_fullsized":"2024-09-21T02:04:08.107333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836206+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:03:59.836206+0000","last_peered":"2024-09-21T02:03:59.836206+0000","last_clean":"2024-09-21T02:03:59.836206+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:03:59.836206+0000","last_undegraded":"2024-09-21T02:03:59.836206+0000","last_fullsized":"2024-09-21T02:03:59.836206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.103523+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:04:08.103523+0000","last_peered":"2024-09-21T02:04:08.103523+0000","last_clean":"2024-09-21T02:04:08.103523+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:04:08.103523+0000","last_undegraded":"2024-09-21T02:04:08.103523+0000","last_fullsized":"2024-09-21T02:04:08.103523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.781583+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:03:59.781583+0000","last_peered":"2024-09-21T02:03:59.781583+0000","last_clean":"2024-09-21T02:03:59.781583+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:03:59.781583+0000","last_undegraded":"2024-09-21T02:03:59.781583+0000","last_fullsized":"2024-09-21T02:03:59.781583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.107012+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:04:08.107012+0000","last_peered":"2024-09-21T02:04:08.107012+0000","last_clean":"2024-09-21T02:04:08.107012+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:04:08.107012+0000","last_undegraded":"2024-09-21T02:04:08.107012+0000","last_fullsized":"2024-09-21T02:04:08.107012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835669+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:03:59.835669+0000","last_peered":"2024-09-21T02:03:59.835669+0000","last_clean":"2024-09-21T02:03:59.835669+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:03:59.835669+0000","last_undegraded":"2024-09-21T02:03:59.835669+0000","last_fullsized":"2024-09-21T02:03:59.835669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.072911+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:03:59.072911+0000","last_peered":"2024-09-21T02:03:59.072911+0000","last_clean":"2024-09-21T02:03:59.072911+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:03:59.072911+0000","last_undegraded":"2024-09-21T02:03:59.072911+0000","last_fullsized":"2024-09-21T02:03:59.072911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836615+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:03:59.836615+0000","last_peered":"2024-09-21T02:03:59.836615+0000","last_clean":"2024-09-21T02:03:59.836615+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:03:59.836615+0000","last_undegraded":"2024-09-21T02:03:59.836615+0000","last_fullsized":"2024-09-21T02:03:59.836615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:08.104423+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:04:08.104423+0000","last_peered":"2024-09-21T02:04:08.104423+0000","last_clean":"2024-09-21T02:04:08.104423+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:04:08.104423+0000","last_undegraded":"2024-09-21T02:04:08.104423+0000","last_fullsized":"2024-09-21T02:04:08.104423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.835473+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:03:59.835473+0000","last_peered":"2024-09-21T02:03:59.835473+0000","last_clean":"2024-09-21T02:03:59.835473+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:03:59.835473+0000","last_undegraded":"2024-09-21T02:03:59.835473+0000","last_fullsized":"2024-09-21T02:03:59.835473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.076421+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:03:59.076421+0000","last_peered":"2024-09-21T02:03:59.076421+0000","last_clean":"2024-09-21T02:03:59.076421+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:03:59.076421+0000","last_undegraded":"2024-09-21T02:03:59.076421+0000","last_fullsized":"2024-09-21T02:03:59.076421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069209+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:03.069209+0000","last_peered":"2024-09-21T02:04:03.069209+0000","last_clean":"2024-09-21T02:04:03.069209+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:03.069209+0000","last_undegraded":"2024-09-21T02:04:03.069209+0000","last_fullsized":"2024-09-21T02:04:03.069209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.075820+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:03:59.075820+0000","last_peered":"2024-09-21T02:03:59.075820+0000","last_clean":"2024-09-21T02:03:59.075820+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:03:59.075820+0000","last_undegraded":"2024-09-21T02:03:59.075820+0000","last_fullsized":"2024-09-21T02:03:59.075820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.137573+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:04:15.137573+0000","last_peered":"2024-09-21T02:04:15.137573+0000","last_clean":"2024-09-21T02:04:15.137573+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:04:15.137573+0000","last_undegraded":"2024-09-21T02:04:15.137573+0000","last_fullsized":"2024-09-21T02:04:15.137573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.836754+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:03:59.836754+0000","last_peered":"2024-09-21T02:03:59.836754+0000","last_clean":"2024-09-21T02:03:59.836754+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:03:59.836754+0000","last_undegraded":"2024-09-21T02:03:59.836754+0000","last_fullsized":"2024-09-21T02:03:59.836754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:03.069853+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:04:03.069853+0000","last_peered":"2024-09-21T02:04:03.069853+0000","last_clean":"2024-09-21T02:04:03.069853+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:04:03.069853+0000","last_undegraded":"2024-09-21T02:04:03.069853+0000","last_fullsized":"2024-09-21T02:04:03.069853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:03:59.837178+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:03:59.837178+0000","last_peered":"2024-09-21T02:03:59.837178+0000","last_clean":"2024-09-21T02:03:59.837178+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:03:59.837178+0000","last_undegraded":"2024-09-21T02:03:59.837178+0000","last_fullsized":"2024-09-21T02:03:59.837178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T02:04:15.140116+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:04:15.140116+0000","last_peered":"2024-09-21T02:04:15.140116+0000","last_clean":"2024-09-21T02:04:15.140116+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:04:15.140116+0000","last_undegraded":"2024-09-21T02:04:15.140116+0000","last_fullsized":"2024-09-21T02:04:15.140116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5734,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":823296,"data_stored":664431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":74125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":73717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":75327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":73565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":73691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35568,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336272,"statfs":{"total":96636764160,"available":96600342528,"internally_reserved":0,"allocated":229376,"data_stored":75549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":663775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:04:27.351 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-21T02:04:27.352 DEBUG:tasks.mon_thrash:Yielding 2024-09-21T02:04:27.352 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T02:04:27.369 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:04:27.369 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T02:04:27.369 INFO:tasks.workunit:timeout=3h 2024-09-21T02:04:27.370 INFO:tasks.workunit:cleanup=True 2024-09-21T02:04:27.370 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:27.370 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:04:27.388 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:04:27.392 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 29h/41d Inode: 1 Links: 2 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 02:04:03.060224473 +0000 2024-09-21T02:04:27.404 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 02:04:23.669668646 +0000 2024-09-21T02:04:27.405 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 02:04:03.060224473 +0000 2024-09-21T02:04:27.405 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T02:04:27.405 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T02:04:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.504+0000 7f50617cc640 1 -- 172.21.15.88:0/723627062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 msgr2=0x7f50480040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.504+0000 7f50617cc640 1 --2- 172.21.15.88:0/723627062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 0x7f50480040b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f504c006ef0 tx=0x7f5048005830 comp rx=0 tx=0).stop 2024-09-21T02:04:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.504+0000 7f505b7fe640 1 -- 172.21.15.88:0/723627062 reap_dead start 2024-09-21T02:04:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.504+0000 7f50617cc640 1 -- 172.21.15.88:0/723627062 shutdown_connections 2024-09-21T02:04:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.504+0000 7f50617cc640 1 -- 172.21.15.88:0/723627062 >> 172.21.15.88:0/723627062 conn(0x7f505c0ca9b0 msgr2=0x7f505c0cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.504+0000 7f50617cc640 1 -- 172.21.15.88:0/723627062 shutdown_connections 2024-09-21T02:04:27.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.504+0000 7f50617cc640 1 -- 172.21.15.88:0/723627062 wait complete. 2024-09-21T02:04:27.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.506+0000 7f50617cc640 1 Processor -- start 2024-09-21T02:04:27.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.506+0000 7f50617cc640 1 -- start start 2024-09-21T02:04:27.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f50617cc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 0x7f505c0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f50617cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f505c0c5170 0x7f505c0c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f50617cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f505c06cf40 0x7f505c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f50617cc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f505c0ca650 con 0x7f5048003c60 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f50617cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f505c074490 con 0x7f505c0c5170 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f50617cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f505c0ca7d0 con 0x7f505c06cf40 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f505affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 0x7f505c0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f505affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 0x7f505c0c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49798/0 (socket says 172.21.15.88:49798) 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f505affd640 1 -- 172.21.15.88:0/2142062237 learned_addr learned my addr 172.21.15.88:0/2142062237 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f505b7fe640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f505c06cf40 0x7f505c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.507+0000 7f5053fff640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f505c0c5170 0x7f505c0c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f505affd640 1 -- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f505c06cf40 msgr2=0x7f505c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f505affd640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f505c06cf40 0x7f505c06d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f505affd640 1 -- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f505c0c5170 msgr2=0x7f505c0c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f505affd640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f505c0c5170 0x7f505c0c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f505affd640 1 -- 172.21.15.88:0/2142062237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505c06d890 con 0x7f5048003c60 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f505affd640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 0x7f505c0c65b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f504c006ef0 tx=0x7f504c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f50537fe640 1 -- 172.21.15.88:0/2142062237 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c001bb0 con 0x7f5048003c60 2024-09-21T02:04:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f50537fe640 1 -- 172.21.15.88:0/2142062237 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f504c03f070 con 0x7f5048003c60 2024-09-21T02:04:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f50537fe640 1 -- 172.21.15.88:0/2142062237 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c003d90 con 0x7f5048003c60 2024-09-21T02:04:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505c15a870 con 0x7f5048003c60 2024-09-21T02:04:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.508+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505c1577b0 con 0x7f5048003c60 2024-09-21T02:04:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.509+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505c157a90 con 0x7f5048003c60 2024-09-21T02:04:27.555 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:04:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.510+0000 7f50537fe640 1 -- 172.21.15.88:0/2142062237 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f504c044470 con 0x7f5048003c60 2024-09-21T02:04:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.510+0000 7f50537fe640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f503c078b90 0x7f503c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.510+0000 7f50537fe640 1 -- 172.21.15.88:0/2142062237 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f504c0bbc90 con 0x7f5048003c60 2024-09-21T02:04:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.510+0000 7f5053fff640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f503c078b90 0x7f503c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.511+0000 7f5053fff640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f503c078b90 0x7f503c07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f505c146820 tx=0x7f5044005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.515+0000 7f50537fe640 1 -- 172.21.15.88:0/2142062237 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f504c083a80 con 0x7f5048003c60 2024-09-21T02:04:27.611 INFO:tasks.workunit.client.0.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T02:04:27.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.734+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f505c0c9ee0 con 0x7f5048003c60 2024-09-21T02:04:27.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.735+0000 7f50537fe640 1 -- 172.21.15.88:0/2142062237 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6908 (secure 0 0 0) 0x7f505c0c9ee0 con 0x7f5048003c60 2024-09-21T02:04:27.736 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:04:27.736 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":13,"btime":"2024-09-21T02:04:16:141092+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:16.141080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4398,4399,4401]},"id":1}]} 2024-09-21T02:04:27.736 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 13 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f503c078b90 msgr2=0x7f503c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f503c078b90 0x7f503c07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f505c146820 tx=0x7f5044005f90 comp rx=0 tx=0).stop 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 msgr2=0x7f505c0c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 0x7f505c0c65b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f504c006ef0 tx=0x7f504c0017a0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 shutdown_connections 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f503c078b90 0x7f503c07b0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f505c06cf40 0x7f505c06d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f505c0c5170 0x7f505c0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 --2- 172.21.15.88:0/2142062237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5048003c60 0x7f505c0c65b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 >> 172.21.15.88:0/2142062237 conn(0x7f505c0ca9b0 msgr2=0x7f505c071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 shutdown_connections 2024-09-21T02:04:27.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.737+0000 7f50617cc640 1 -- 172.21.15.88:0/2142062237 wait complete. 2024-09-21T02:04:27.750 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-21T02:04:27.750 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726884267, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-21T02:04:27.750 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-21T02:04:27.750 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:04:27.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 -- 172.21.15.88:0/900285870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c0c41a0 msgr2=0x7f1d9c0c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 --2- 172.21.15.88:0/900285870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c0c41a0 0x7f1d9c0c45b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0c6080 tx=0x7f1d9402a8b0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 -- 172.21.15.88:0/900285870 shutdown_connections 2024-09-21T02:04:27.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 --2- 172.21.15.88:0/900285870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c0c41a0 0x7f1d9c0c45b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 --2- 172.21.15.88:0/900285870 >> v2:172.21.15.195:3300/0 conn(0x7f1d9c153320 0x7f1d9c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 -- 172.21.15.88:0/900285870 >> 172.21.15.88:0/900285870 conn(0x7f1d9c073210 msgr2=0x7f1d9c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:27.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 -- 172.21.15.88:0/900285870 shutdown_connections 2024-09-21T02:04:27.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.900+0000 7f1da38a9640 1 -- 172.21.15.88:0/900285870 wait complete. 2024-09-21T02:04:27.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.902+0000 7f1da38a9640 1 Processor -- start 2024-09-21T02:04:27.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.902+0000 7f1da38a9640 1 -- start start 2024-09-21T02:04:27.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da38a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9c0c41a0 0x7f1d9c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da38a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9c0c99a0 0x7f1d9c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da38a9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c153320 0x7f1d9c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da38a9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c070ae0 con 0x7f1d9c153320 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da38a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c072240 con 0x7f1d9c0c99a0 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da38a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9c070dc0 con 0x7f1d9c0c41a0 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da1e46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c153320 0x7f1d9c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da1e46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c153320 0x7f1d9c154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49826/0 (socket says 172.21.15.88:49826) 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.903+0000 7f1da1e46640 1 -- 172.21.15.88:0/2390966373 learned_addr learned my addr 172.21.15.88:0/2390966373 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1645640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9c0c41a0 0x7f1d9c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da0e44640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9c0c99a0 0x7f1d9c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1e46640 1 -- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9c0c41a0 msgr2=0x7f1d9c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1e46640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9c0c41a0 0x7f1d9c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1e46640 1 -- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9c0c99a0 msgr2=0x7f1d9c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:27.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1e46640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9c0c99a0 0x7f1d9c15b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1e46640 1 -- 172.21.15.88:0/2390966373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c1f0300 con 0x7f1d9c153320 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1645640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9c0c41a0 0x7f1d9c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da0e44640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9c0c99a0 0x7f1d9c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da1e46640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c153320 0x7f1d9c154440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005fd0 tx=0x7f1d94003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1d91ffb640 1 -- 172.21.15.88:0/2390966373 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94031640 con 0x7f1d9c153320 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1d91ffb640 1 -- 172.21.15.88:0/2390966373 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d94003e40 con 0x7f1d9c153320 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c1f05e0 con 0x7f1d9c153320 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1d91ffb640 1 -- 172.21.15.88:0/2390966373 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940332d0 con 0x7f1d9c153320 2024-09-21T02:04:27.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.904+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c0ce4f0 con 0x7f1d9c153320 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.905+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c15c400 con 0x7f1d9c153320 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.906+0000 7f1d91ffb640 1 -- 172.21.15.88:0/2390966373 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d94033470 con 0x7f1d9c153320 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.906+0000 7f1d91ffb640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1d80078b90 0x7f1d8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.906+0000 7f1d91ffb640 1 -- 172.21.15.88:0/2390966373 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1d940bc340 con 0x7f1d9c153320 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.907+0000 7f1da1645640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1d80078b90 0x7f1d8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:27.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.907+0000 7f1da1645640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1d80078b90 0x7f1d8007b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007fd0 tx=0x7f1d98009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:27.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:27.911+0000 7f1d91ffb640 1 -- 172.21.15.88:0/2390966373 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d9c15c400 con 0x7f1d9c153320 2024-09-21T02:04:28.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.159+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1d9c15afe0 con 0x7f1d9c153320 2024-09-21T02:04:28.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.159+0000 7f1d91ffb640 1 -- 172.21.15.88:0/2390966373 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f1d9c15afe0 con 0x7f1d9c153320 2024-09-21T02:04:28.160 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":55,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:04:28.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1d80078b90 msgr2=0x7f1d8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1d80078b90 0x7f1d8007b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007fd0 tx=0x7f1d98009040 comp rx=0 tx=0).stop 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c153320 msgr2=0x7f1d9c154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c153320 0x7f1d9c154440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005fd0 tx=0x7f1d94003c20 comp rx=0 tx=0).stop 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 shutdown_connections 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1d80078b90 0x7f1d8007b0a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d9c153320 0x7f1d9c154440 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1d9c0c99a0 0x7f1d9c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 --2- 172.21.15.88:0/2390966373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1d9c0c41a0 0x7f1d9c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 >> 172.21.15.88:0/2390966373 conn(0x7f1d9c073210 msgr2=0x7f1d9c0cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 shutdown_connections 2024-09-21T02:04:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.161+0000 7f1da38a9640 1 -- 172.21.15.88:0/2390966373 wait complete. 2024-09-21T02:04:28.173 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-21T02:04:28.174 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-21T02:04:28.174 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-21T02:04:28.174 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:04:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.286+0000 7fe7391c6640 1 -- 172.21.15.88:0/3890492969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 msgr2=0x7fe720005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.286+0000 7fe7391c6640 1 --2- 172.21.15.88:0/3890492969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 0x7fe720005150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c610 tx=0x7fe720005830 comp rx=0 tx=0).stop 2024-09-21T02:04:28.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.286+0000 7fe73359e640 1 -- 172.21.15.88:0/3890492969 reap_dead start 2024-09-21T02:04:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.286+0000 7fe7391c6640 1 -- 172.21.15.88:0/3890492969 shutdown_connections 2024-09-21T02:04:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.286+0000 7fe7391c6640 1 -- 172.21.15.88:0/3890492969 >> 172.21.15.88:0/3890492969 conn(0x7fe7340d16d0 msgr2=0x7fe7340cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.286+0000 7fe7391c6640 1 -- 172.21.15.88:0/3890492969 shutdown_connections 2024-09-21T02:04:28.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.286+0000 7fe7391c6640 1 -- 172.21.15.88:0/3890492969 wait complete. 2024-09-21T02:04:28.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.288+0000 7fe7391c6640 1 Processor -- start 2024-09-21T02:04:28.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.288+0000 7fe7391c6640 1 -- start start 2024-09-21T02:04:28.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.289+0000 7fe7391c6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 0x7fe7341f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.289+0000 7fe7391c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe734156c30 0x7fe7341533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.289+0000 7fe7391c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73414c420 0x7fe73414c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.289+0000 7fe7391c6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe73415bab0 con 0x7fe720004d40 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.289+0000 7fe7391c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe73415c3a0 con 0x7fe734156c30 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.289+0000 7fe7391c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe7340b48b0 con 0x7fe73414c420 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.289+0000 7fe732d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 0x7fe7341f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 0x7fe7341f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37620/0 (socket says 172.21.15.88:37620) 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 -- 172.21.15.88:0/631175167 learned_addr learned my addr 172.21.15.88:0/631175167 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe73359e640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73414c420 0x7fe73414c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe72a59c640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe734156c30 0x7fe7341533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73414c420 msgr2=0x7fe73414c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe73414c420 0x7fe73414c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe734156c30 msgr2=0x7fe7341533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe734156c30 0x7fe7341533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 -- 172.21.15.88:0/631175167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7341f5b60 con 0x7fe720004d40 2024-09-21T02:04:28.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe732d9d640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 0x7fe7341f1600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe724011ae0 tx=0x7fe724011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72400fc40 con 0x7fe720004d40 2024-09-21T02:04:28.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe72400fde0 con 0x7fe720004d40 2024-09-21T02:04:28.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73414d130 con 0x7fe720004d40 2024-09-21T02:04:28.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724004280 con 0x7fe720004d40 2024-09-21T02:04:28.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.290+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7341f4da0 con 0x7fe720004d40 2024-09-21T02:04:28.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.291+0000 7fe7391c6640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7341f1be0 0x7fe7341f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.291+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341f5520 con 0x7fe7341f1be0 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.291+0000 7fe72a59c640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7341f1be0 0x7fe7341f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.292+0000 7fe72a59c640 1 -- 172.21.15.88:0/631175167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7340cf220 con 0x7fe7341f1be0 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.292+0000 7fe72a59c640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7341f1be0 0x7fe7341f4fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7341ebf70 tx=0x7fe71c012e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.292+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c004020 con 0x7fe7341f1be0 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.292+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe724020020 con 0x7fe720004d40 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe72bfff640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe714078a90 0x7fe71407afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe72409b470 con 0x7fe720004d40 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe71c0041c0 con 0x7fe7341f1be0 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe73359e640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe714078a90 0x7fe71407afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7341f1be0 msgr2=0x7fe7341f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe72bfff640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7341f1be0 0x7fe7341f4fe0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe7341ebf70 tx=0x7fe71c012e80 comp rx=0 tx=0).stop 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7341f1be0 msgr2=0x7fe7341f4fe0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:28.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe72bfff640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7341f1be0 0x7fe7341f4fe0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.293+0000 7fe73359e640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe714078a90 0x7fe71407afa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe72c033000 tx=0x7fe72c005040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.299+0000 7fe7391c6640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe734150b90 0x7fe734150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.299+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe73415c8c0 con 0x7fe734150b90 2024-09-21T02:04:28.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.299+0000 7fe72a59c640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe734150b90 0x7fe734150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.299+0000 7fe72a59c640 1 -- 172.21.15.88:0/631175167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe734159960 con 0x7fe734150b90 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe72a59c640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe734150b90 0x7fe734150fa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe71c01b000 tx=0x7fe71c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c005050 con 0x7fe734150b90 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4884 (secure 0 0 0) 0x7fe71c005050 con 0x7fe734150b90 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe734150b90 msgr2=0x7fe734150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe72bfff640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe734150b90 0x7fe734150fa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe71c01b000 tx=0x7fe71c004730 comp rx=0 tx=0).stop 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe72bfff640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe734150b90 msgr2=0x7fe734150fa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe72bfff640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe734150b90 0x7fe734150fa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 4, 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 56016, 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:04:28.302 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 55, 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.303 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.304 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:28.305 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.306 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:04:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.309 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.310 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.311 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.312 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.313 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:04:28.314 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:28.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe714078a90 msgr2=0x7fe71407afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe714078a90 0x7fe71407afa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe72c033000 tx=0x7fe72c005040 comp rx=0 tx=0).stop 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 msgr2=0x7fe7341f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 --2- 172.21.15.88:0/631175167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe720004d40 0x7fe7341f1600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe724011ae0 tx=0x7fe724011ee0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe73359e640 1 -- 172.21.15.88:0/631175167 reap_dead start 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe73359e640 1 -- 172.21.15.88:0/631175167 reap_dead start 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 shutdown_connections 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 >> 172.21.15.88:0/631175167 conn(0x7fe7340d16d0 msgr2=0x7fe734155550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 shutdown_connections 2024-09-21T02:04:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.300+0000 7fe7391c6640 1 -- 172.21.15.88:0/631175167 wait complete. 2024-09-21T02:04:28.316 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:04:28.316 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-21T02:04:28.317 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:04:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 -- 172.21.15.88:0/2328854972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c7af0 msgr2=0x7f2d000c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 --2- 172.21.15.88:0/2328854972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c7af0 0x7f2d000c8c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8005fd0 tx=0x7f2ce802a670 comp rx=0 tx=0).stop 2024-09-21T02:04:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 -- 172.21.15.88:0/2328854972 shutdown_connections 2024-09-21T02:04:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 --2- 172.21.15.88:0/2328854972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d000c91d0 0x7f2d0006de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 --2- 172.21.15.88:0/2328854972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c7af0 0x7f2d000c8c90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 -- 172.21.15.88:0/2328854972 >> 172.21.15.88:0/2328854972 conn(0x7f2d000720b0 msgr2=0x7f2d000cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 -- 172.21.15.88:0/2328854972 shutdown_connections 2024-09-21T02:04:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.429+0000 7f2d04901640 1 -- 172.21.15.88:0/2328854972 wait complete. 2024-09-21T02:04:28.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.431+0000 7f2d04901640 1 Processor -- start 2024-09-21T02:04:28.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.431+0000 7f2d04901640 1 -- start start 2024-09-21T02:04:28.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2d04901640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c2e60 0x7f2d00149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2d04901640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d000c7af0 0x7f2d000c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2d04901640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d000c91d0 0x7f2d0014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2d04901640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00159190 con 0x7f2d000c2e60 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2d04901640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00159480 con 0x7f2d000c91d0 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2d04901640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0006f1f0 con 0x7f2d000c7af0 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2cfe59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c2e60 0x7f2d00149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2cfe59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c2e60 0x7f2d00149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37666/0 (socket says 172.21.15.88:37666) 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2cfe59c640 1 -- 172.21.15.88:0/4144499886 learned_addr learned my addr 172.21.15.88:0/4144499886 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2cfdd9b640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d000c7af0 0x7f2d000c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.432+0000 7f2cfed9d640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d000c91d0 0x7f2d0014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cfe59c640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d000c7af0 msgr2=0x7f2d000c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cfe59c640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d000c7af0 0x7f2d000c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cfe59c640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d000c91d0 msgr2=0x7f2d0014b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cfe59c640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d000c91d0 0x7f2d0014b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cfe59c640 1 -- 172.21.15.88:0/4144499886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d001eef50 con 0x7f2d000c2e60 2024-09-21T02:04:28.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cfed9d640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d000c91d0 0x7f2d0014b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:04:28.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cfe59c640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c2e60 0x7f2d00149e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000c7a0 tx=0x7f2cf000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0017070 con 0x7f2d000c2e60 2024-09-21T02:04:28.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cf0005070 con 0x7f2d000c2e60 2024-09-21T02:04:28.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0014a3b0 con 0x7f2d000c2e60 2024-09-21T02:04:28.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00054a0 con 0x7f2d000c2e60 2024-09-21T02:04:28.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.433+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d00148710 con 0x7f2d000c2e60 2024-09-21T02:04:28.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.434+0000 7f2d04901640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d0014c550 0x7f2d000cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.434+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000c8c90 con 0x7f2d0014c550 2024-09-21T02:04:28.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.434+0000 7f2cfdd9b640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d0014c550 0x7f2d000cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.434+0000 7f2cfdd9b640 1 -- 172.21.15.88:0/4144499886 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d000cc390 con 0x7f2d0014c550 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.435+0000 7f2cfdd9b640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d0014c550 0x7f2d000cbe50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ce4a0 tx=0x7f2ce80035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.435+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8040070 con 0x7f2d0014c550 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.435+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2cf001a070 con 0x7f2d000c2e60 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.435+0000 7f2cf6ffd640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2ce0078a90 0x7f2ce007afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2cf009a270 con 0x7f2d000c2e60 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cfed9d640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2ce0078a90 0x7f2ce007afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2ce8003c80 con 0x7f2d0014c550 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d0014c550 msgr2=0x7f2d000cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cf6ffd640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d0014c550 0x7f2d000cbe50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ce4a0 tx=0x7f2ce80035f0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d0014c550 msgr2=0x7f2d000cbe50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cf6ffd640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d0014c550 0x7f2d000cbe50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.436+0000 7f2cfed9d640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2ce0078a90 0x7f2ce007afa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2d0014f220 tx=0x7f2cf8005e30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.442+0000 7f2d04901640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d00147920 0x7f2d00147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.442+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00148270 con 0x7f2d00147920 2024-09-21T02:04:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.442+0000 7f2cfdd9b640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d00147920 0x7f2d00147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cfdd9b640 1 -- 172.21.15.88:0/4144499886 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2d0014f810 con 0x7f2d00147920 2024-09-21T02:04:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cfdd9b640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d00147920 0x7f2d00147d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8006010 tx=0x7f2ce8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8035040 con 0x7f2d00147920 2024-09-21T02:04:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5025 (secure 0 0 0) 0x7f2ce803cc60 con 0x7f2d00147920 2024-09-21T02:04:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d00147920 msgr2=0x7f2d00147d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cf6ffd640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d00147920 0x7f2d00147d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8006010 tx=0x7f2ce8009260 comp rx=0 tx=0).stop 2024-09-21T02:04:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cf6ffd640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d00147920 msgr2=0x7f2d00147d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.443+0000 7f2cf6ffd640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d00147920 0x7f2d00147d30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 4, 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 56116, 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.445 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 55, 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.446 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:04:28.448 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.449 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.450 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.451 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.452 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.453 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.454 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.455 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.456 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:04:28.457 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:28.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2ce0078a90 msgr2=0x7f2ce007afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2ce0078a90 0x7f2ce007afa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2d0014f220 tx=0x7f2cf8005e30 comp rx=0 tx=0).stop 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c2e60 msgr2=0x7f2d00149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 --2- 172.21.15.88:0/4144499886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d000c2e60 0x7f2d00149e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000c7a0 tx=0x7f2cf000cc60 comp rx=0 tx=0).stop 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2cfed9d640 1 -- 172.21.15.88:0/4144499886 reap_dead start 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2cfed9d640 1 -- 172.21.15.88:0/4144499886 reap_dead start 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 shutdown_connections 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 >> 172.21.15.88:0/4144499886 conn(0x7f2d000720b0 msgr2=0x7f2d000c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 shutdown_connections 2024-09-21T02:04:28.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.444+0000 7f2d04901640 1 -- 172.21.15.88:0/4144499886 wait complete. 2024-09-21T02:04:28.459 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:04:28.459 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-21T02:04:28.460 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 -- 172.21.15.88:0/1419905270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b18f0 msgr2=0x7f12e4155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/1419905270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b18f0 0x7f12e4155c70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f12e40c4230 tx=0x7f12d402a560 comp rx=0 tx=0).stop 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 -- 172.21.15.88:0/1419905270 shutdown_connections 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/1419905270 >> v2:172.21.15.195:3300/0 conn(0x7f12e40c28d0 0x7f12e40c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/1419905270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b18f0 0x7f12e4155c70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 -- 172.21.15.88:0/1419905270 >> 172.21.15.88:0/1419905270 conn(0x7f12e406d8f0 msgr2=0x7f12e40ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 -- 172.21.15.88:0/1419905270 shutdown_connections 2024-09-21T02:04:28.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.571+0000 7f12ea6a2640 1 -- 172.21.15.88:0/1419905270 wait complete. 2024-09-21T02:04:28.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.573+0000 7f12ea6a2640 1 Processor -- start 2024-09-21T02:04:28.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.573+0000 7f12ea6a2640 1 -- start start 2024-09-21T02:04:28.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.574+0000 7f12ea6a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e40b18f0 0x7f12e4073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12ea6a2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b1e60 0x7f12e40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12ea6a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e40c28d0 0x7f12e40753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12ea6a2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4071230 con 0x7f12e40b1e60 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12ea6a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f12e40c9600 con 0x7f12e40c28d0 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12ea6a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12e415b960 con 0x7f12e40b18f0 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b1e60 0x7f12e40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b1e60 0x7f12e40cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37676/0 (socket says 172.21.15.88:37676) 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 -- 172.21.15.88:0/2629223827 learned_addr learned my addr 172.21.15.88:0/2629223827 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e3fff640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e40b18f0 0x7f12e4073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e8c3f640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e40c28d0 0x7f12e40753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e40b18f0 msgr2=0x7f12e4073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e40b18f0 0x7f12e4073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e40c28d0 msgr2=0x7f12e40753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e40c28d0 0x7f12e40753b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 -- 172.21.15.88:0/2629223827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e4074000 con 0x7f12e40b1e60 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e37fe640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b1e60 0x7f12e40cc450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c3d0 tx=0x7f12d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8017070 con 0x7f12e40b1e60 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12d80053c0 con 0x7f12e40b1e60 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e40cc990 con 0x7f12e40b1e60 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8003850 con 0x7f12e40b1e60 2024-09-21T02:04:28.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.575+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e41f63e0 con 0x7f12e40b1e60 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.576+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4152730 0x7f12e41e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.576+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12e40c1ec0 con 0x7f12e4152730 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.576+0000 7f12e3fff640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4152730 0x7f12e41e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.577+0000 7f12e3fff640 1 -- 172.21.15.88:0/2629223827 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12e415c5b0 con 0x7f12e4152730 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.577+0000 7f12e3fff640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4152730 0x7f12e41e6680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cad50 tx=0x7f12d4004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.577+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4040070 con 0x7f12e4152730 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.577+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12d801a070 con 0x7f12e40b1e60 2024-09-21T02:04:28.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e0ff9640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f12d0078a90 0x7f12d007afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f12d809a0b0 con 0x7f12e40b1e60 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e8c3f640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f12d0078a90 0x7f12d007afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f12d4003bc0 con 0x7f12e4152730 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4152730 msgr2=0x7f12e41e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e0ff9640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4152730 0x7f12e41e6680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cad50 tx=0x7f12d4004570 comp rx=0 tx=0).stop 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4152730 msgr2=0x7f12e41e6680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e0ff9640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4152730 0x7f12e41e6680 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cad50 tx=0x7f12d4004570 comp rx=0 tx=0).stop 2024-09-21T02:04:28.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.578+0000 7f12e8c3f640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f12d0078a90 0x7f12d007afa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12e4152e80 tx=0x7f12dc009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.584+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4154570 0x7f12e4154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.584+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f12e40c3360 con 0x7f12e4154570 2024-09-21T02:04:28.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.584+0000 7f12e3fff640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4154570 0x7f12e4154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e3fff640 1 -- 172.21.15.88:0/2629223827 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f12e415ad70 con 0x7f12e4154570 2024-09-21T02:04:28.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e3fff640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4154570 0x7f12e4154980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000c00 tx=0x7f12d4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4035040 con 0x7f12e4154570 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5049 (secure 0 0 0) 0x7f12d403c850 con 0x7f12e4154570 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4154570 msgr2=0x7f12e4154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e0ff9640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4154570 0x7f12e4154980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000c00 tx=0x7f12d4009260 comp rx=0 tx=0).stop 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e0ff9640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4154570 msgr2=0x7f12e4154980 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.585+0000 7f12e0ff9640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e4154570 0x7f12e4154980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:04:28.587 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 4, 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 56258, 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 56, 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:28.588 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.589 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.590 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:04:28.591 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:28.592 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:04:28.593 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.594 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.595 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:04:28.596 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.597 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.598 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:04:28.599 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:04:28.600 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:04:28.601 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:28.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f12d0078a90 msgr2=0x7f12d007afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f12d0078a90 0x7f12d007afa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12e4152e80 tx=0x7f12dc009500 comp rx=0 tx=0).stop 2024-09-21T02:04:28.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b1e60 msgr2=0x7f12e40cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b1e60 0x7f12e40cc450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c3d0 tx=0x7f12d800c890 comp rx=0 tx=0).stop 2024-09-21T02:04:28.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12e8c3f640 1 -- 172.21.15.88:0/2629223827 reap_dead start 2024-09-21T02:04:28.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 shutdown_connections 2024-09-21T02:04:28.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 --2- 172.21.15.88:0/2629223827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e40b1e60 0x7f12e40cc450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 >> 172.21.15.88:0/2629223827 conn(0x7f12e406d8f0 msgr2=0x7f12e406ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 shutdown_connections 2024-09-21T02:04:28.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.586+0000 7f12ea6a2640 1 -- 172.21.15.88:0/2629223827 wait complete. 2024-09-21T02:04:28.603 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:04:28.603 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-21T02:04:28.604 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-21T02:04:28.604 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-21T02:04:28.604 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-21T02:04:28.604 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T02:04:28.604 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:04:28.621 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T02:04:28.621 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-21T02:04:28.621 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:04:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.731+0000 7f312658b640 1 -- 172.21.15.88:0/828762268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3120153320 msgr2=0x7f3120153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.731+0000 7f312658b640 1 --2- 172.21.15.88:0/828762268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3120153320 0x7f3120153730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3114005fd0 tx=0x7f311402a5d0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.731+0000 7f312658b640 1 -- 172.21.15.88:0/828762268 shutdown_connections 2024-09-21T02:04:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.731+0000 7f312658b640 1 --2- 172.21.15.88:0/828762268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3120153320 0x7f3120153730 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.731+0000 7f312658b640 1 -- 172.21.15.88:0/828762268 >> 172.21.15.88:0/828762268 conn(0x7f3120073210 msgr2=0x7f31200cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.731+0000 7f312658b640 1 -- 172.21.15.88:0/828762268 shutdown_connections 2024-09-21T02:04:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.731+0000 7f312658b640 1 -- 172.21.15.88:0/828762268 wait complete. 2024-09-21T02:04:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.733+0000 7f312658b640 1 Processor -- start 2024-09-21T02:04:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.733+0000 7f312658b640 1 -- start start 2024-09-21T02:04:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f312658b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200c41a0 0x7f31200cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f312658b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31200c99a0 0x7f312015b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f312658b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3120153320 0x7f31200c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f312658b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3120070dc0 con 0x7f31200c99a0 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f312658b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3120070ae0 con 0x7f3120153320 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f312658b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f31200c4cc0 con 0x7f31200c41a0 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f311f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31200c99a0 0x7f312015b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f311f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31200c99a0 0x7f312015b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37710/0 (socket says 172.21.15.88:37710) 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f311f7fe640 1 -- 172.21.15.88:0/3773557986 learned_addr learned my addr 172.21.15.88:0/3773557986 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f311ffff640 1 -- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200c41a0 msgr2=0x7f31200cd130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f311ffff640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200c41a0 0x7f31200cd130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.734+0000 7f3124b28640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3120153320 0x7f31200c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311f7fe640 1 -- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200c41a0 msgr2=0x7f31200cd130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311f7fe640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200c41a0 0x7f31200cd130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311f7fe640 1 -- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3120153320 msgr2=0x7f31200c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311f7fe640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3120153320 0x7f31200c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311f7fe640 1 -- 172.21.15.88:0/3773557986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200cd670 con 0x7f31200c99a0 2024-09-21T02:04:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311f7fe640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31200c99a0 0x7f312015b300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3114034040 tx=0x7f31140038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311cff9640 1 -- 172.21.15.88:0/3773557986 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311403e030 con 0x7f31200c99a0 2024-09-21T02:04:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311cff9640 1 -- 172.21.15.88:0/3773557986 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3114003ca0 con 0x7f31200c99a0 2024-09-21T02:04:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f311cff9640 1 -- 172.21.15.88:0/3773557986 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114035040 con 0x7f31200c99a0 2024-09-21T02:04:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31200c9040 con 0x7f31200c99a0 2024-09-21T02:04:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.735+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31201e4a90 con 0x7f31200c99a0 2024-09-21T02:04:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.736+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31200ca8c0 con 0x7f31200c99a0 2024-09-21T02:04:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.737+0000 7f311cff9640 1 -- 172.21.15.88:0/3773557986 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3114041030 con 0x7f31200c99a0 2024-09-21T02:04:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.737+0000 7f311cff9640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f310c078b40 0x7f310c07b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.737+0000 7f311cff9640 1 -- 172.21.15.88:0/3773557986 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f31140bb400 con 0x7f31200c99a0 2024-09-21T02:04:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.737+0000 7f311ffff640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f310c078b40 0x7f310c07b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.738+0000 7f311ffff640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f310c078b40 0x7f310c07b050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3110003680 tx=0x7f3110007a10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.742+0000 7f311cff9640 1 -- 172.21.15.88:0/3773557986 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3114083840 con 0x7f31200c99a0 2024-09-21T02:04:28.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.984+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f31200c9210 con 0x7f31200c99a0 2024-09-21T02:04:28.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.985+0000 7f311cff9640 1 -- 172.21.15.88:0/3773557986 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f3114083140 con 0x7f31200c99a0 2024-09-21T02:04:28.986 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":56,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:04:28.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f310c078b40 msgr2=0x7f310c07b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f310c078b40 0x7f310c07b050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3110003680 tx=0x7f3110007a10 comp rx=0 tx=0).stop 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31200c99a0 msgr2=0x7f312015b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31200c99a0 0x7f312015b300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3114034040 tx=0x7f31140038a0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 shutdown_connections 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f310c078b40 0x7f310c07b050 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3120153320 0x7f31200c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31200c99a0 0x7f312015b300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 --2- 172.21.15.88:0/3773557986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31200c41a0 0x7f31200cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 >> 172.21.15.88:0/3773557986 conn(0x7f3120073210 msgr2=0x7f31201575b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 shutdown_connections 2024-09-21T02:04:28.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:28.987+0000 7f312658b640 1 -- 172.21.15.88:0/3773557986 wait complete. 2024-09-21T02:04:32.000 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:04:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.127+0000 7fd6acece640 1 -- 172.21.15.88:0/497326636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd694004d90 msgr2=0x7fd694005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.127+0000 7fd6acece640 1 --2- 172.21.15.88:0/497326636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd694004d90 0x7fd694005260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6a806ffa0 tx=0x7fd694005830 comp rx=0 tx=0).stop 2024-09-21T02:04:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.127+0000 7fd6a6d9d640 1 -- 172.21.15.88:0/497326636 reap_dead start 2024-09-21T02:04:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.127+0000 7fd6acece640 1 -- 172.21.15.88:0/497326636 shutdown_connections 2024-09-21T02:04:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.127+0000 7fd6acece640 1 -- 172.21.15.88:0/497326636 >> 172.21.15.88:0/497326636 conn(0x7fd6a8070c30 msgr2=0x7fd6a8071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.127+0000 7fd6acece640 1 -- 172.21.15.88:0/497326636 shutdown_connections 2024-09-21T02:04:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.127+0000 7fd6acece640 1 -- 172.21.15.88:0/497326636 wait complete. 2024-09-21T02:04:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.129+0000 7fd6acece640 1 Processor -- start 2024-09-21T02:04:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.129+0000 7fd6acece640 1 -- start start 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6acece640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd694004d90 0x7fd6a81e1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6acece640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a8156950 0x7fd6a81e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6acece640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a80c55b0 0x7fd6a814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6acece640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a80c32d0 con 0x7fd6a80c55b0 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6acece640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8073d20 con 0x7fd6a8156950 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6acece640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a80c3c00 con 0x7fd694004d90 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a80c55b0 0x7fd6a814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a659c640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd694004d90 msgr2=0x7fd6a81e1160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a80c55b0 0x7fd6a814c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37726/0 (socket says 172.21.15.88:37726) 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 -- 172.21.15.88:0/2594338112 learned_addr learned my addr 172.21.15.88:0/2594338112 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:32.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a659c640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd694004d90 0x7fd6a81e1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a5d9b640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a8156950 0x7fd6a81e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 -- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd694004d90 msgr2=0x7fd6a81e1160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd694004d90 0x7fd6a81e1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 -- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a8156950 msgr2=0x7fd6a81e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a8156950 0x7fd6a81e2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 -- 172.21.15.88:0/2594338112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a81e16a0 con 0x7fd6a80c55b0 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6a6d9d640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a80c55b0 0x7fd6a814c3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0035040 tx=0x7fd6a0032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd69effd640 1 -- 172.21.15.88:0/2594338112 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0037040 con 0x7fd6a80c55b0 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd69effd640 1 -- 172.21.15.88:0/2594338112 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a0041070 con 0x7fd6a80c55b0 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd69effd640 1 -- 172.21.15.88:0/2594338112 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00095a0 con 0x7fd6a80c55b0 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.131+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a81577d0 con 0x7fd6a80c55b0 2024-09-21T02:04:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.132+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a80b2770 con 0x7fd6a80c55b0 2024-09-21T02:04:32.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.133+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a8075910 con 0x7fd6a80c55b0 2024-09-21T02:04:32.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.133+0000 7fd69effd640 1 -- 172.21.15.88:0/2594338112 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6a003b960 con 0x7fd6a80c55b0 2024-09-21T02:04:32.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.134+0000 7fd69effd640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd688078b40 0x7fd68807b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:32.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.134+0000 7fd69effd640 1 -- 172.21.15.88:0/2594338112 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd6a00ba820 con 0x7fd6a80c55b0 2024-09-21T02:04:32.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.134+0000 7fd6a659c640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd688078b40 0x7fd68807b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:32.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.135+0000 7fd6a659c640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd688078b40 0x7fd68807b050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd698005c40 tx=0x7fd698005bd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:32.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.139+0000 7fd69effd640 1 -- 172.21.15.88:0/2594338112 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6a0082cf0 con 0x7fd6a80c55b0 2024-09-21T02:04:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.387+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd6a806a900 con 0x7fd6a80c55b0 2024-09-21T02:04:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.387+0000 7fd69effd640 1 -- 172.21.15.88:0/2594338112 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7fd6a00825f0 con 0x7fd6a80c55b0 2024-09-21T02:04:32.388 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":59,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:04:32.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd688078b40 msgr2=0x7fd68807b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd688078b40 0x7fd68807b050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd698005c40 tx=0x7fd698005bd0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a80c55b0 msgr2=0x7fd6a814c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a80c55b0 0x7fd6a814c3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0035040 tx=0x7fd6a0032e00 comp rx=0 tx=0).stop 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 shutdown_connections 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd688078b40 0x7fd68807b050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6a80c55b0 0x7fd6a814c3f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6a8156950 0x7fd6a81e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 --2- 172.21.15.88:0/2594338112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd694004d90 0x7fd6a81e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 >> 172.21.15.88:0/2594338112 conn(0x7fd6a8070c30 msgr2=0x7fd6a80cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 shutdown_connections 2024-09-21T02:04:32.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:32.389+0000 7fd6acece640 1 -- 172.21.15.88:0/2594338112 wait complete. 2024-09-21T02:04:35.402 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:04:35.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/133379650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280ccd00 msgr2=0x7f7b2815f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/133379650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280ccd00 0x7f7b2815f970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005fd0 tx=0x7f7b1802a640 comp rx=0 tx=0).stop 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/133379650 shutdown_connections 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/133379650 >> v2:172.21.15.195:3300/0 conn(0x7f7b280cad60 0x7f7b2806cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/133379650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b280cb170 0x7f7b2806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/133379650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280ccd00 0x7f7b2815f970 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/133379650 >> 172.21.15.88:0/133379650 conn(0x7f7b280cd9f0 msgr2=0x7f7b280cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/133379650 shutdown_connections 2024-09-21T02:04:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.526+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/133379650 wait complete. 2024-09-21T02:04:35.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.528+0000 7f7b2d7ad640 1 Processor -- start 2024-09-21T02:04:35.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.528+0000 7f7b2d7ad640 1 -- start start 2024-09-21T02:04:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b2d7ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b280cad60 0x7f7b281e6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b2d7ad640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280cb170 0x7f7b281f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b2d7ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b280ccd00 0x7f7b280b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b2d7ad640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2806dcd0 con 0x7f7b280cb170 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b2d7ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2815c640 con 0x7f7b280cad60 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b2d7ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2806de50 con 0x7f7b280ccd00 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b267fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280cb170 0x7f7b281f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b267fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280cb170 0x7f7b281f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37738/0 (socket says 172.21.15.88:37738) 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b267fc640 1 -- 172.21.15.88:0/1362544344 learned_addr learned my addr 172.21.15.88:0/1362544344 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.529+0000 7f7b277fe640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b280ccd00 msgr2=0x7f7b280b5640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b277fe640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b280ccd00 0x7f7b280b5640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b26ffd640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b280cad60 0x7f7b281e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b267fc640 1 -- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b280ccd00 msgr2=0x7f7b280b5640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b267fc640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b280ccd00 0x7f7b280b5640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b267fc640 1 -- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b280cad60 msgr2=0x7f7b281e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b267fc640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b280cad60 0x7f7b281e6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b267fc640 1 -- 172.21.15.88:0/1362544344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b281f2dc0 con 0x7f7b280cb170 2024-09-21T02:04:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b267fc640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280cb170 0x7f7b281f2880 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00e790 tx=0x7f7b1c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b0f7fe640 1 -- 172.21.15.88:0/1362544344 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c00eea0 con 0x7f7b280cb170 2024-09-21T02:04:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b0f7fe640 1 -- 172.21.15.88:0/1362544344 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b1c00cdb0 con 0x7f7b280cb170 2024-09-21T02:04:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b281e5d10 con 0x7f7b280cb170 2024-09-21T02:04:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b0f7fe640 1 -- 172.21.15.88:0/1362544344 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c010690 con 0x7f7b280cb170 2024-09-21T02:04:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.530+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281f2360 con 0x7f7b280cb170 2024-09-21T02:04:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.531+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b281f5670 con 0x7f7b280cb170 2024-09-21T02:04:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.532+0000 7f7b0f7fe640 1 -- 172.21.15.88:0/1362544344 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b1c010830 con 0x7f7b280cb170 2024-09-21T02:04:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.533+0000 7f7b0f7fe640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7b14078b70 0x7f7b1407b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.533+0000 7f7b0f7fe640 1 -- 172.21.15.88:0/1362544344 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7b1c09b6e0 con 0x7f7b280cb170 2024-09-21T02:04:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.533+0000 7f7b26ffd640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7b14078b70 0x7f7b1407b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.533+0000 7f7b26ffd640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7b14078b70 0x7f7b1407b080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b18033000 tx=0x7f7b18004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:35.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.537+0000 7f7b0f7fe640 1 -- 172.21.15.88:0/1362544344 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7b1c0634d0 con 0x7f7b280cb170 2024-09-21T02:04:35.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.785+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7b281e6230 con 0x7f7b280cb170 2024-09-21T02:04:35.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.785+0000 7f7b0f7fe640 1 -- 172.21.15.88:0/1362544344 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f7b1c067f00 con 0x7f7b280cb170 2024-09-21T02:04:35.786 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":63,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7b14078b70 msgr2=0x7f7b1407b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7b14078b70 0x7f7b1407b080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b18033000 tx=0x7f7b18004730 comp rx=0 tx=0).stop 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280cb170 msgr2=0x7f7b281f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280cb170 0x7f7b281f2880 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00e790 tx=0x7f7b1c00ec50 comp rx=0 tx=0).stop 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 shutdown_connections 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7b14078b70 0x7f7b1407b080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b280ccd00 0x7f7b280b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b280cb170 0x7f7b281f2880 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.787+0000 7f7b2d7ad640 1 --2- 172.21.15.88:0/1362544344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b280cad60 0x7f7b281e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:35.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.788+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 >> 172.21.15.88:0/1362544344 conn(0x7f7b280cd9f0 msgr2=0x7f7b281553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:35.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.788+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 shutdown_connections 2024-09-21T02:04:35.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:35.788+0000 7f7b2d7ad640 1 -- 172.21.15.88:0/1362544344 wait complete. 2024-09-21T02:04:38.800 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:04:43.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 -- 172.21.15.88:0/2133981248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0004800 msgr2=0x7f27c0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2133981248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0004800 0x7f27c0004c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27c4006f10 tx=0x7f27c0005830 comp rx=0 tx=0).stop 2024-09-21T02:04:43.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 -- 172.21.15.88:0/2133981248 shutdown_connections 2024-09-21T02:04:43.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2133981248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27c0005350 0x7f27c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2133981248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0004800 0x7f27c0004c10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2133981248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27c00039b0 0x7f27c0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 -- 172.21.15.88:0/2133981248 >> 172.21.15.88:0/2133981248 conn(0x7f27d4070c30 msgr2=0x7f27d4071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:43.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 -- 172.21.15.88:0/2133981248 shutdown_connections 2024-09-21T02:04:43.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.196+0000 7f27dbc84640 1 -- 172.21.15.88:0/2133981248 wait complete. 2024-09-21T02:04:43.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.198+0000 7f27dbc84640 1 Processor -- start 2024-09-21T02:04:43.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.198+0000 7f27dbc84640 1 -- start start 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27dbc84640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27c00039b0 0x7f27d41eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27dbc84640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27c0004800 0x7f27d415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27dbc84640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0005350 0x7f27d414d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27dbc84640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40c3370 con 0x7f27c0005350 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27dbc84640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40c3090 con 0x7f27c00039b0 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27dbc84640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f27d4075070 con 0x7f27c0004800 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27d921f640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27c0004800 msgr2=0x7f27d415aef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27d921f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27c0004800 0x7f27d415aef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:04:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27da221640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0005350 0x7f27d414d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27da221640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0005350 0x7f27d414d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39188/0 (socket says 172.21.15.88:39188) 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27da221640 1 -- 172.21.15.88:0/2403290352 learned_addr learned my addr 172.21.15.88:0/2403290352 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.199+0000 7f27d9a20640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27c00039b0 0x7f27d41eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27da221640 1 -- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27c0004800 msgr2=0x7f27d415aef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27da221640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27c0004800 0x7f27d415aef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27da221640 1 -- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27c00039b0 msgr2=0x7f27d41eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27da221640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27c00039b0 0x7f27d41eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27da221640 1 -- 172.21.15.88:0/2403290352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d41ef480 con 0x7f27c0005350 2024-09-21T02:04:43.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27da221640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0005350 0x7f27d414d490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f27cc013710 tx=0x7f27cc013b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27ca7fc640 1 -- 172.21.15.88:0/2403290352 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc011d70 con 0x7f27c0005350 2024-09-21T02:04:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27ca7fc640 1 -- 172.21.15.88:0/2403290352 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27cc004b20 con 0x7f27c0005350 2024-09-21T02:04:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27ca7fc640 1 -- 172.21.15.88:0/2403290352 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc005020 con 0x7f27c0005350 2024-09-21T02:04:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d415a220 con 0x7f27c0005350 2024-09-21T02:04:43.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.200+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d40c4890 con 0x7f27c0005350 2024-09-21T02:04:43.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.202+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d40c8e40 con 0x7f27c0005350 2024-09-21T02:04:43.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.202+0000 7f27ca7fc640 1 -- 172.21.15.88:0/2403290352 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27cc00c040 con 0x7f27c0005350 2024-09-21T02:04:43.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.203+0000 7f27ca7fc640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f27b8079470 0x7f27b807b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.203+0000 7f27ca7fc640 1 -- 172.21.15.88:0/2403290352 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f27cc09c210 con 0x7f27c0005350 2024-09-21T02:04:43.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.203+0000 7f27d9a20640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f27b8079470 0x7f27b807b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.204+0000 7f27d9a20640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f27b8079470 0x7f27b807b980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001050 tx=0x7f27d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.208+0000 7f27ca7fc640 1 -- 172.21.15.88:0/2403290352 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27cc021060 con 0x7f27c0005350 2024-09-21T02:04:43.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.455+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f27d40cc8d0 con 0x7f27c0005350 2024-09-21T02:04:43.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.455+0000 7f27ca7fc640 1 -- 172.21.15.88:0/2403290352 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f27cc0646e0 con 0x7f27c0005350 2024-09-21T02:04:43.456 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":8,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f27b8079470 msgr2=0x7f27b807b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f27b8079470 0x7f27b807b980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001050 tx=0x7f27d0009040 comp rx=0 tx=0).stop 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0005350 msgr2=0x7f27d414d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0005350 0x7f27d414d490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f27cc013710 tx=0x7f27cc013b10 comp rx=0 tx=0).stop 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 shutdown_connections 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f27b8079470 0x7f27b807b980 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c0005350 0x7f27d414d490 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27c0004800 0x7f27d415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 --2- 172.21.15.88:0/2403290352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27c00039b0 0x7f27d41eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 >> 172.21.15.88:0/2403290352 conn(0x7f27d4070c30 msgr2=0x7f27d4076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.457+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 shutdown_connections 2024-09-21T02:04:43.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.458+0000 7f27dbc84640 1 -- 172.21.15.88:0/2403290352 wait complete. 2024-09-21T02:04:43.470 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-21T02:04:43.470 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-21T02:04:43.470 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:04:43.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.583+0000 7f0d16225640 1 -- 172.21.15.88:0/948919876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cf8004f20 msgr2=0x7f0cf800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.583+0000 7f0d16225640 1 --2- 172.21.15.88:0/948919876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cf8004f20 0x7f0cf800b9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d101573f0 tx=0x7f0cf8005830 comp rx=0 tx=0).stop 2024-09-21T02:04:43.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.583+0000 7f0d0ffff640 1 -- 172.21.15.88:0/948919876 reap_dead start 2024-09-21T02:04:43.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.583+0000 7f0d16225640 1 -- 172.21.15.88:0/948919876 shutdown_connections 2024-09-21T02:04:43.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.583+0000 7f0d16225640 1 -- 172.21.15.88:0/948919876 >> 172.21.15.88:0/948919876 conn(0x7f0d10070960 msgr2=0x7f0d1006cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:43.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.583+0000 7f0d16225640 1 -- 172.21.15.88:0/948919876 shutdown_connections 2024-09-21T02:04:43.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.583+0000 7f0d16225640 1 -- 172.21.15.88:0/948919876 wait complete. 2024-09-21T02:04:43.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.585+0000 7f0d16225640 1 Processor -- start 2024-09-21T02:04:43.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.585+0000 7f0d16225640 1 -- start start 2024-09-21T02:04:43.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d16225640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101557d0 0x7f0d101efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d16225640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d101566c0 0x7f0d100cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d16225640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d1014f980 0x7f0d1014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d16225640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10159ed0 con 0x7f0d101557d0 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d16225640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1015a240 con 0x7f0d101566c0 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d16225640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1015a6d0 con 0x7f0d1014f980 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d0ffff640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d1014f980 msgr2=0x7f0d1014fd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d0ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d1014f980 0x7f0d1014fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.586+0000 7f0d0f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101557d0 0x7f0d101efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101557d0 0x7f0d101efbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39210/0 (socket says 172.21.15.88:39210) 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 -- 172.21.15.88:0/1963383327 learned_addr learned my addr 172.21.15.88:0/1963383327 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0effd640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d101566c0 0x7f0d100cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d1014f980 msgr2=0x7f0d1014fd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d1014f980 0x7f0d1014fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d101566c0 msgr2=0x7f0d100cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d101566c0 0x7f0d100cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 -- 172.21.15.88:0/1963383327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d101502d0 con 0x7f0d101557d0 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d0f7fe640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101557d0 0x7f0d101efbb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000efc0 tx=0x7f0d0000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00018020 con 0x7f0d101557d0 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d00014480 con 0x7f0d101557d0 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d10152d80 con 0x7f0d101557d0 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d000148d0 con 0x7f0d101557d0 2024-09-21T02:04:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.587+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d101e58c0 con 0x7f0d101557d0 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.588+0000 7f0d16225640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d10155d20 0x7f0d101f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.588+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1006a3d0 con 0x7f0d10155d20 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.588+0000 7f0d0effd640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d10155d20 0x7f0d101f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.588+0000 7f0d0effd640 1 -- 172.21.15.88:0/1963383327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d101e5390 con 0x7f0d10155d20 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.589+0000 7f0d0effd640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d10155d20 0x7f0d101f31a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d100c9440 tx=0x7f0d0400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.589+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04019070 con 0x7f0d10155d20 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.589+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d0001b020 con 0x7f0d101557d0 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.589+0000 7f0cf7fff640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0cfc078a90 0x7f0cfc07afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0d0009bb00 con 0x7f0d101557d0 2024-09-21T02:04:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0d04005110 con 0x7f0d10155d20 2024-09-21T02:04:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d10155d20 msgr2=0x7f0d101f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0cf7fff640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d10155d20 0x7f0d101f31a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d100c9440 tx=0x7f0d0400fc00 comp rx=0 tx=0).stop 2024-09-21T02:04:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d10155d20 msgr2=0x7f0d101f31a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0cf7fff640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d10155d20 0x7f0d101f31a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0d0ffff640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0cfc078a90 0x7f0cfc07afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.590+0000 7f0d0ffff640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0cfc078a90 0x7f0cfc07afa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d08033000 tx=0x7f0d0802d000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.596+0000 7f0d16225640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101f36e0 0x7f0d101f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.596+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d101f4030 con 0x7f0d101f36e0 2024-09-21T02:04:43.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.596+0000 7f0d0effd640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101f36e0 0x7f0d101f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.596+0000 7f0d0effd640 1 -- 172.21.15.88:0/1963383327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0d101e5ac0 con 0x7f0d101f36e0 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.596+0000 7f0d0effd640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101f36e0 0x7f0d101f3af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0d100c9440 tx=0x7f0d04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.596+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04006c60 con 0x7f0d101f36e0 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5039 (secure 0 0 0) 0x7f0d04006c60 con 0x7f0d101f36e0 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101f36e0 msgr2=0x7f0d101f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0cf7fff640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101f36e0 0x7f0d101f3af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0d100c9440 tx=0x7f0d04000bc0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0cf7fff640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101f36e0 msgr2=0x7f0d101f3af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0cf7fff640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101f36e0 0x7f0d101f3af0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.598 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:43.599 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:04:43.599 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:04:43.601 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:04:43.601 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 8, 2024-09-21T02:04:43.601 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 71313, 2024-09-21T02:04:43.601 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:04:43.601 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:04:43.601 INFO:teuthology.orchestra.run.smithi088.stdout: 1 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:43.602 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:43.603 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:43.604 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:43.605 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:43.606 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:04:43.607 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:43.608 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:43.609 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:04:43.610 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:04:43.611 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.612 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:04:43.613 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:43.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0cfc078a90 msgr2=0x7f0cfc07afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0cfc078a90 0x7f0cfc07afa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d08033000 tx=0x7f0d0802d000 comp rx=0 tx=0).stop 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101557d0 msgr2=0x7f0d101efbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 --2- 172.21.15.88:0/1963383327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d101557d0 0x7f0d101efbb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000efc0 tx=0x7f0d0000c490 comp rx=0 tx=0).stop 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d0ffff640 1 -- 172.21.15.88:0/1963383327 reap_dead start 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d0ffff640 1 -- 172.21.15.88:0/1963383327 reap_dead start 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 shutdown_connections 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 >> 172.21.15.88:0/1963383327 conn(0x7f0d10070960 msgr2=0x7f0d100c81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 shutdown_connections 2024-09-21T02:04:43.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.597+0000 7f0d16225640 1 -- 172.21.15.88:0/1963383327 wait complete. 2024-09-21T02:04:43.615 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-21T02:04:43.616 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-21T02:04:43.616 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 -- 172.21.15.88:0/3196092319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd600cd140 msgr2=0x7fbd600caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 --2- 172.21.15.88:0/3196092319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd600cd140 0x7fbd600caf30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbd50005fd0 tx=0x7fbd5002a5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 -- 172.21.15.88:0/3196092319 shutdown_connections 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 --2- 172.21.15.88:0/3196092319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd6015f070 0x7fbd60074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 --2- 172.21.15.88:0/3196092319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd600cd140 0x7fbd600caf30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 -- 172.21.15.88:0/3196092319 >> 172.21.15.88:0/3196092319 conn(0x7fbd600d16b0 msgr2=0x7fbd600cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 -- 172.21.15.88:0/3196092319 shutdown_connections 2024-09-21T02:04:43.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.727+0000 7fbd65007640 1 -- 172.21.15.88:0/3196092319 wait complete. 2024-09-21T02:04:43.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.729+0000 7fbd65007640 1 Processor -- start 2024-09-21T02:04:43.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.729+0000 7fbd65007640 1 -- start start 2024-09-21T02:04:43.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd65007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd600749d0 0x7fbd601f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd65007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd600cd140 0x7fbd60156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd65007640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6015f070 0x7fbd600b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd65007640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbd600cb470 con 0x7fbd6015f070 2024-09-21T02:04:43.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd65007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd600cbb30 con 0x7fbd600cd140 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd65007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6015bb00 con 0x7fbd600749d0 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5ed9d640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd600749d0 msgr2=0x7fbd601f5150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5ed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd600749d0 0x7fbd601f5150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6015f070 0x7fbd600b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6015f070 0x7fbd600b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39242/0 (socket says 172.21.15.88:39242) 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 -- 172.21.15.88:0/276995516 learned_addr learned my addr 172.21.15.88:0/276995516 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5659c640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd600cd140 0x7fbd60156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd600749d0 msgr2=0x7fbd601f5150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbd600749d0 0x7fbd601f5150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd600cd140 msgr2=0x7fbd60156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd600cd140 0x7fbd60156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 -- 172.21.15.88:0/276995516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd60157890 con 0x7fbd6015f070 2024-09-21T02:04:43.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd5f59e640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6015f070 0x7fbd600b5230 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c730 tx=0x7fbd5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58003b70 con 0x7fbd6015f070 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd58003d10 con 0x7fbd6015f070 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.731+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd601f5690 con 0x7fbd6015f070 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.732+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800aab0 con 0x7fbd6015f070 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.732+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd601f1f80 con 0x7fbd6015f070 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.732+0000 7fbd65007640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f58b0 0x7fbd600b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.732+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd60157b60 con 0x7fbd601f58b0 2024-09-21T02:04:43.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.732+0000 7fbd5ed9d640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f58b0 0x7fbd600b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.733+0000 7fbd5ed9d640 1 -- 172.21.15.88:0/276995516 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd600ce880 con 0x7fbd601f58b0 2024-09-21T02:04:43.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.733+0000 7fbd5ed9d640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f58b0 0x7fbd600b5770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd6008f0a0 tx=0x7fbd50004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.733+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd5800ac50 con 0x7fbd6015f070 2024-09-21T02:04:43.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbd40078b70 0x7fbd4007b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50035040 con 0x7fbd601f58b0 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd5809aab0 con 0x7fbd6015f070 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbd50004220 con 0x7fbd601f58b0 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f58b0 msgr2=0x7fbd600b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f58b0 0x7fbd600b5770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd6008f0a0 tx=0x7fbd50004d70 comp rx=0 tx=0).stop 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f58b0 msgr2=0x7fbd600b5770 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd57fff640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f58b0 0x7fbd600b5770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.734+0000 7fbd5659c640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbd40078b70 0x7fbd4007b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.735+0000 7fbd5659c640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbd40078b70 0x7fbd4007b080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd60156b80 tx=0x7fbd48009600 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.740+0000 7fbd65007640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f21c0 0x7fbd60158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:04:43.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.740+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd600c97b0 con 0x7fbd601f21c0 2024-09-21T02:04:43.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.740+0000 7fbd5ed9d640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f21c0 0x7fbd60158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:04:43.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.741+0000 7fbd5ed9d640 1 -- 172.21.15.88:0/276995516 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbd601f2920 con 0x7fbd601f21c0 2024-09-21T02:04:43.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.741+0000 7fbd5ed9d640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f21c0 0x7fbd60158ee0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd50005fd0 tx=0x7fbd5002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:04:43.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.741+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50035070 con 0x7fbd601f21c0 2024-09-21T02:04:43.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.741+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5013 (secure 0 0 0) 0x7fbd50031ca0 con 0x7fbd601f21c0 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.741+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f21c0 msgr2=0x7fbd60158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.741+0000 7fbd57fff640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f21c0 0x7fbd60158ee0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd50005fd0 tx=0x7fbd5002a5f0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd57fff640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f21c0 msgr2=0x7fbd60158ee0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd57fff640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd601f21c0 0x7fbd60158ee0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 8, 2024-09-21T02:04:43.743 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 71415, 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: 1 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:43.744 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:43.745 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:04:43.746 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:04:43.747 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:04:43.748 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:04:43.749 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:04:43.750 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.751 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.752 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.753 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.754 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:04:43.755 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:04:43.756 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:04:43.756 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbd40078b70 msgr2=0x7fbd4007b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbd40078b70 0x7fbd4007b080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd60156b80 tx=0x7fbd48009600 comp rx=0 tx=0).stop 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6015f070 msgr2=0x7fbd600b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 --2- 172.21.15.88:0/276995516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6015f070 0x7fbd600b5230 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c730 tx=0x7fbd5800cbf0 comp rx=0 tx=0).stop 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd5f59e640 1 -- 172.21.15.88:0/276995516 reap_dead start 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd5f59e640 1 -- 172.21.15.88:0/276995516 reap_dead start 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 shutdown_connections 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 >> 172.21.15.88:0/276995516 conn(0x7fbd600d16b0 msgr2=0x7fbd600c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 shutdown_connections 2024-09-21T02:04:43.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:04:43.742+0000 7fbd65007640 1 -- 172.21.15.88:0/276995516 wait complete. 2024-09-21T02:04:43.758 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-21T02:04:43.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-21T02:04:57.459 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:04:57.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:05:03.761 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-21T02:05:03.762 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T02:05:03.762 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T02:05:03.765 INFO:tasks.ceph.mon.c:Started 2024-09-21T02:05:03.765 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-21T02:05:03.765 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 -- 172.21.15.88:0/2197395255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 msgr2=0x7f750406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2197395255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f750406d2a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f750408e110 tx=0x7f74fc02aac0 comp rx=0 tx=0).stop 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 -- 172.21.15.88:0/2197395255 shutdown_connections 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2197395255 >> v2:172.21.15.195:3300/0 conn(0x7f75040cad60 0x7f750406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2197395255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f750406d2a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2197395255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 0x7f750415f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 -- 172.21.15.88:0/2197395255 >> 172.21.15.88:0/2197395255 conn(0x7f75040cd9f0 msgr2=0x7f75040cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 -- 172.21.15.88:0/2197395255 shutdown_connections 2024-09-21T02:05:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.876+0000 7f750c6d5640 1 -- 172.21.15.88:0/2197395255 wait complete. 2024-09-21T02:05:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.878+0000 7f750c6d5640 1 Processor -- start 2024-09-21T02:05:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.878+0000 7f750c6d5640 1 -- start start 2024-09-21T02:05:03.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.879+0000 7f750c6d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040cad60 0x7f75041f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.879+0000 7f750c6d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f75041f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.879+0000 7f750c6d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 0x7f75041e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.879+0000 7f750c6d5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f750406dcd0 con 0x7f75040cb170 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.879+0000 7f750c6d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f750415c640 con 0x7f75040cad60 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.879+0000 7f750c6d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f750406de50 con 0x7f75040ccd00 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f75041f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f75041f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35322/0 (socket says 172.21.15.88:35322) 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f750a471640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040cad60 0x7f75041f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 -- 172.21.15.88:0/2432887562 learned_addr learned my addr 172.21.15.88:0/2432887562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f750ac72640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 0x7f75041e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:03.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f750ac72640 1 -- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 msgr2=0x7f75041e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f750ac72640 1 -- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 msgr2=0x7f75041e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f750ac72640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 0x7f75041e62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f750ac72640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 0x7f75041e62d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 -- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 msgr2=0x7f75041e62d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 0x7f75041e62d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 -- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040cad60 msgr2=0x7f75041f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040cad60 0x7f75041f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 -- 172.21.15.88:0/2432887562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75041f6070 con 0x7f75040cb170 2024-09-21T02:05:03.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f7509c70640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f75041f5b30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e7c0 tx=0x7f74f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:03.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f74faffd640 1 -- 172.21.15.88:0/2432887562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f400cc60 con 0x7f75040cb170 2024-09-21T02:05:03.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f74faffd640 1 -- 172.21.15.88:0/2432887562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74f4010430 con 0x7f75040cb170 2024-09-21T02:05:03.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f74faffd640 1 -- 172.21.15.88:0/2432887562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4010860 con 0x7f75040cb170 2024-09-21T02:05:03.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.880+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750415e440 con 0x7f75040cb170 2024-09-21T02:05:03.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.881+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750415ef40 con 0x7f75040cb170 2024-09-21T02:05:03.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.882+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750415f970 con 0x7f75040cb170 2024-09-21T02:05:03.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.883+0000 7f74faffd640 1 -- 172.21.15.88:0/2432887562 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74f4010a00 con 0x7f75040cb170 2024-09-21T02:05:03.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.883+0000 7f74faffd640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f74e8078bc0 0x7f74e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:03.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.884+0000 7f74faffd640 1 -- 172.21.15.88:0/2432887562 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f74f409b5b0 con 0x7f75040cb170 2024-09-21T02:05:03.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.884+0000 7f750a471640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f74e8078bc0 0x7f74e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:03.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.884+0000 7f750a471640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f74e8078bc0 0x7f74e807b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7500006b00 tx=0x7f7500006a90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:03.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:03.888+0000 7f74faffd640 1 -- 172.21.15.88:0/2432887562 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f74f4063440 con 0x7f75040cb170 2024-09-21T02:05:04.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.311+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7504155f30 con 0x7f75040cb170 2024-09-21T02:05:04.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.311+0000 7f74faffd640 1 -- 172.21.15.88:0/2432887562 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f74f4067e70 con 0x7f75040cb170 2024-09-21T02:05:04.313 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:05:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.314+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f74e8078bc0 msgr2=0x7f74e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.314+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f74e8078bc0 0x7f74e807b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7500006b00 tx=0x7f7500006a90 comp rx=0 tx=0).stop 2024-09-21T02:05:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.314+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 msgr2=0x7f75041f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.314+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f75041f5b30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e7c0 tx=0x7f74f400ec80 comp rx=0 tx=0).stop 2024-09-21T02:05:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 shutdown_connections 2024-09-21T02:05:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f74e8078bc0 0x7f74e807b0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75040ccd00 0x7f75041e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75040cb170 0x7f75041f5b30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 --2- 172.21.15.88:0/2432887562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75040cad60 0x7f75041f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 >> 172.21.15.88:0/2432887562 conn(0x7f75040cd9f0 msgr2=0x7f7504155460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 shutdown_connections 2024-09-21T02:05:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.315+0000 7f750c6d5640 1 -- 172.21.15.88:0/2432887562 wait complete. 2024-09-21T02:05:04.330 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-21T02:05:04.330 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-21T02:05:04.330 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:05:04.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.447+0000 7f74123c6640 1 -- 172.21.15.88:0/727157648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73f4005350 msgr2=0x7f73f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.447+0000 7f74123c6640 1 --2- 172.21.15.88:0/727157648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73f4005350 0x7f73f400b9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f740c072550 tx=0x7f73f4005830 comp rx=0 tx=0).stop 2024-09-21T02:05:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.447+0000 7f74123c6640 1 -- 172.21.15.88:0/727157648 shutdown_connections 2024-09-21T02:05:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.447+0000 7f74123c6640 1 --2- 172.21.15.88:0/727157648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73f4005350 0x7f73f400b9c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.447+0000 7f74123c6640 1 -- 172.21.15.88:0/727157648 >> 172.21.15.88:0/727157648 conn(0x7f740c0729d0 msgr2=0x7f740c072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.447+0000 7f74123c6640 1 -- 172.21.15.88:0/727157648 shutdown_connections 2024-09-21T02:05:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.447+0000 7f74123c6640 1 -- 172.21.15.88:0/727157648 wait complete. 2024-09-21T02:05:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.449+0000 7f74123c6640 1 Processor -- start 2024-09-21T02:05:04.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.449+0000 7f74123c6640 1 -- start start 2024-09-21T02:05:04.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f74123c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f4005350 0x7f740c1efc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f74123c6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c1e4490 0x7f740c150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f74123c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 0x7f740c1e5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f74123c6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f740c0c4280 con 0x7f740c1e4490 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f74123c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f740c070ab0 con 0x7f73f4005350 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f740b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c1e4490 0x7f740c150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f74123c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f740c15a2f0 con 0x7f740c1e5310 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f740b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c1e4490 0x7f740c150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35348/0 (socket says 172.21.15.88:35348) 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f740b7fe640 1 -- 172.21.15.88:0/2942032467 learned_addr learned my addr 172.21.15.88:0/2942032467 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f740bfff640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f4005350 0x7f740c1efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.451+0000 7f7410963640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 0x7f740c1e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f7410963640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 msgr2=0x7f740c1e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f7410963640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 msgr2=0x7f740c1e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f7410963640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 0x7f740c1e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f7410963640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 0x7f740c1e5720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f740b7fe640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 msgr2=0x7f740c1e5720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f740b7fe640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f740c1e5310 0x7f740c1e5720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f740b7fe640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f4005350 msgr2=0x7f740c1efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f740b7fe640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f4005350 0x7f740c1efc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f740b7fe640 1 -- 172.21.15.88:0/2942032467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c1f0160 con 0x7f740c1e4490 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f740b7fe640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c1e4490 0x7f740c150c00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7400011ec0 tx=0x7f74000039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400001670 con 0x7f740c1e4490 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7400001810 con 0x7f740c1e4490 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400001c20 con 0x7f740c1e4490 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c0c6e00 con 0x7f740c1e4490 2024-09-21T02:05:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.452+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c1e4de0 con 0x7f740c1e4490 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.454+0000 7f74123c6640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c0ca320 0x7f740c15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.454+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1f0630 con 0x7f740c0ca320 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.454+0000 7f740bfff640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c0ca320 0x7f740c15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.454+0000 7f740bfff640 1 -- 172.21.15.88:0/2942032467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f740c06fe80 con 0x7f740c0ca320 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.454+0000 7f740bfff640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c0ca320 0x7f740c15a970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f73fc001990 tx=0x7f73fc003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.454+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f740001c430 con 0x7f740c1e4490 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f73f8078b90 0x7f73f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f740009f060 con 0x7f740c1e4490 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0129d0 con 0x7f740c0ca320 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f73fc012b70 con 0x7f740c0ca320 2024-09-21T02:05:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c0ca320 msgr2=0x7f740c15a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c0ca320 0x7f740c15a970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f73fc001990 tx=0x7f73fc003a80 comp rx=0 tx=0).stop 2024-09-21T02:05:04.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c0ca320 msgr2=0x7f740c15a970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:04.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7408ff9640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c0ca320 0x7f740c15a970 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.455+0000 7f7410963640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f73f8078b90 0x7f73f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.456+0000 7f7410963640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f73f8078b90 0x7f73f807b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c3d0 tx=0x7f740402bf90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.467+0000 7f74123c6640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c15b960 0x7f740c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.467+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f740c15c2b0 con 0x7f740c15b960 2024-09-21T02:05:04.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.467+0000 7f740bfff640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c15b960 0x7f740c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f740bfff640 1 -- 172.21.15.88:0/2942032467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f740c0b51a0 con 0x7f740c15b960 2024-09-21T02:05:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f740bfff640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c15b960 0x7f740c15bd70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f73fc001990 tx=0x7f740c1597d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0129d0 con 0x7f740c15b960 2024-09-21T02:05:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5051 (secure 0 0 0) 0x7f73fc00f7f0 con 0x7f740c15b960 2024-09-21T02:05:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c15b960 msgr2=0x7f740c15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f7408ff9640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c15b960 0x7f740c15bd70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f73fc001990 tx=0x7f740c1597d0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f7408ff9640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c15b960 msgr2=0x7f740c15bd70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.468+0000 7f7408ff9640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c15b960 0x7f740c15bd70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 10, 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 92185, 2024-09-21T02:05:04.470 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:04.471 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:04.472 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:04.473 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:04.474 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.475 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:04.476 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:04.477 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:04.478 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.479 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:04.480 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.481 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.482 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:04.483 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f73f8078b90 msgr2=0x7f73f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f73f8078b90 0x7f73f807b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c3d0 tx=0x7f740402bf90 comp rx=0 tx=0).stop 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c1e4490 msgr2=0x7f740c150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 --2- 172.21.15.88:0/2942032467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f740c1e4490 0x7f740c150c00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7400011ec0 tx=0x7f74000039c0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f7410963640 1 -- 172.21.15.88:0/2942032467 reap_dead start 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f7410963640 1 -- 172.21.15.88:0/2942032467 reap_dead start 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 shutdown_connections 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 >> 172.21.15.88:0/2942032467 conn(0x7f740c0729d0 msgr2=0x7f740c0c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 shutdown_connections 2024-09-21T02:05:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.469+0000 7f74123c6640 1 -- 172.21.15.88:0/2942032467 wait complete. 2024-09-21T02:05:04.486 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:05:04.486 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-21T02:05:04.486 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:05:04.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.627+0000 7f6bfe941640 1 -- 172.21.15.88:0/1355106237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6be0003c10 msgr2=0x7f6be0004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.627+0000 7f6bfe941640 1 --2- 172.21.15.88:0/1355106237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6be0003c10 0x7f6be0004060 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c4a0 tx=0x7f6be0005830 comp rx=0 tx=0).stop 2024-09-21T02:05:04.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.627+0000 7f6bfcede640 1 -- 172.21.15.88:0/1355106237 reap_dead start 2024-09-21T02:05:04.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.627+0000 7f6bfe941640 1 -- 172.21.15.88:0/1355106237 shutdown_connections 2024-09-21T02:05:04.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.627+0000 7f6bfe941640 1 -- 172.21.15.88:0/1355106237 >> 172.21.15.88:0/1355106237 conn(0x7f6bf80729d0 msgr2=0x7f6bf8072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.627+0000 7f6bfe941640 1 -- 172.21.15.88:0/1355106237 shutdown_connections 2024-09-21T02:05:04.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.627+0000 7f6bfe941640 1 -- 172.21.15.88:0/1355106237 wait complete. 2024-09-21T02:05:04.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.629+0000 7f6bfe941640 1 Processor -- start 2024-09-21T02:05:04.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.629+0000 7f6bfe941640 1 -- start start 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfe941640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6be0003c10 0x7f6bf815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfe941640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815be20 0x7f6bf81536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfe941640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf815c230 0x7f6bf814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfe941640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf80c4280 con 0x7f6bf815c230 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfe941640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8070ab0 con 0x7f6bf815be20 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfe941640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf815a2f0 con 0x7f6be0003c10 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf815c230 0x7f6bf814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf815c230 0x7f6bf814f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35388/0 (socket says 172.21.15.88:35388) 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 -- 172.21.15.88:0/656428664 learned_addr learned my addr 172.21.15.88:0/656428664 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bf7fff640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6be0003c10 0x7f6bf815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bf77fe640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815be20 0x7f6bf81536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6be0003c10 msgr2=0x7f6bf815ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6be0003c10 0x7f6bf815ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815be20 msgr2=0x7f6bf81536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815be20 0x7f6bf81536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 -- 172.21.15.88:0/656428664 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf815d390 con 0x7f6bf815c230 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.631+0000 7f6bfcede640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf815c230 0x7f6bf814f8c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6bf815de80 tx=0x7f6bf0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.632+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0011900 con 0x7f6bf815c230 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.632+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bf0018070 con 0x7f6bf815c230 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.632+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0011ca0 con 0x7f6bf815c230 2024-09-21T02:05:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.632+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf81507d0 con 0x7f6bf815c230 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.632+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf815aec0 con 0x7f6bf815c230 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.633+0000 7f6bfe941640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815d560 0x7f6bf815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.633+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf815a990 con 0x7f6bf815d560 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.633+0000 7f6bf7fff640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815d560 0x7f6bf815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.634+0000 7f6bf7fff640 1 -- 172.21.15.88:0/656428664 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bf80c9690 con 0x7f6bf815d560 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.634+0000 7f6bf7fff640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815d560 0x7f6bf815b0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c470 tx=0x7f6be80064c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.634+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8043070 con 0x7f6bf815d560 2024-09-21T02:05:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.634+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6bf001d460 con 0x7f6bf815c230 2024-09-21T02:05:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf4ff9640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6be4078ae0 0x7f6be407aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6bf009b5e0 con 0x7f6bf815c230 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6be802f040 con 0x7f6bf815d560 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815d560 msgr2=0x7f6bf815b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf4ff9640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815d560 0x7f6bf815b0c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c470 tx=0x7f6be80064c0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815d560 msgr2=0x7f6bf815b0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf4ff9640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815d560 0x7f6bf815b0c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.635+0000 7f6bf77fe640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6be4078ae0 0x7f6be407aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.636+0000 7f6bf77fe640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6be4078ae0 0x7f6be407aff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0057a0 tx=0x7f6bec005c70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.641+0000 7f6bfe941640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf80ca320 0x7f6bf814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.641+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf80b51a0 con 0x7f6bf80ca320 2024-09-21T02:05:04.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.641+0000 7f6bf7fff640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf80ca320 0x7f6bf814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf7fff640 1 -- 172.21.15.88:0/656428664 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6bf80c5dc0 con 0x7f6bf80ca320 2024-09-21T02:05:04.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf7fff640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf80ca320 0x7f6bf814fe00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6bf815ab10 tx=0x7f6be80068a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8006d90 con 0x7f6bf80ca320 2024-09-21T02:05:04.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5025 (secure 0 0 0) 0x7f6be80066f0 con 0x7f6bf80ca320 2024-09-21T02:05:04.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf80ca320 msgr2=0x7f6bf814fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf4ff9640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf80ca320 0x7f6bf814fe00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6bf815ab10 tx=0x7f6be80068a0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf4ff9640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf80ca320 msgr2=0x7f6bf814fe00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:04.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.642+0000 7f6bf4ff9640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf80ca320 0x7f6bf814fe00 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6bf815ab10 tx=0x7f6be80068a0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.644 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 10, 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 92316, 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:04.646 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:04.647 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:04.648 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:04.649 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:04.650 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:04.651 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:04.652 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:04.653 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.654 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.655 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:04.656 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:05:04.657 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6be4078ae0 msgr2=0x7f6be407aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6be4078ae0 0x7f6be407aff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0057a0 tx=0x7f6bec005c70 comp rx=0 tx=0).stop 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf815c230 msgr2=0x7f6bf814f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 --2- 172.21.15.88:0/656428664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf815c230 0x7f6bf814f8c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6bf815de80 tx=0x7f6bf0006430 comp rx=0 tx=0).stop 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfcede640 1 -- 172.21.15.88:0/656428664 reap_dead start 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfcede640 1 -- 172.21.15.88:0/656428664 reap_dead start 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 shutdown_connections 2024-09-21T02:05:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 >> 172.21.15.88:0/656428664 conn(0x7f6bf80729d0 msgr2=0x7f6bf80cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 shutdown_connections 2024-09-21T02:05:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.643+0000 7f6bfe941640 1 -- 172.21.15.88:0/656428664 wait complete. 2024-09-21T02:05:04.661 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:05:04.661 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-21T02:05:04.661 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-21T02:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.775+0000 7f40dfc84640 1 -- 172.21.15.88:0/1197672623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c91d0 msgr2=0x7f40d806de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.775+0000 7f40dfc84640 1 --2- 172.21.15.88:0/1197672623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c91d0 0x7f40d806de70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f40d806ef30 tx=0x7f40d002ab40 comp rx=0 tx=0).stop 2024-09-21T02:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.776+0000 7f40dfc84640 1 -- 172.21.15.88:0/1197672623 shutdown_connections 2024-09-21T02:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.776+0000 7f40dfc84640 1 --2- 172.21.15.88:0/1197672623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c91d0 0x7f40d806de70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.776+0000 7f40dfc84640 1 --2- 172.21.15.88:0/1197672623 >> v2:172.21.15.195:3300/0 conn(0x7f40d80c7af0 0x7f40d80c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.776+0000 7f40dfc84640 1 -- 172.21.15.88:0/1197672623 >> 172.21.15.88:0/1197672623 conn(0x7f40d80720b0 msgr2=0x7f40d80cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.776+0000 7f40dfc84640 1 -- 172.21.15.88:0/1197672623 shutdown_connections 2024-09-21T02:05:04.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.776+0000 7f40dfc84640 1 -- 172.21.15.88:0/1197672623 wait complete. 2024-09-21T02:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.778+0000 7f40dfc84640 1 Processor -- start 2024-09-21T02:05:04.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.779+0000 7f40dfc84640 1 -- start start 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dfc84640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d80c2e60 0x7f40d814d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dfc84640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c7af0 0x7f40d8147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c7af0 0x7f40d8147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c7af0 0x7f40d8147a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35414/0 (socket says 172.21.15.88:35414) 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dfc84640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80c91d0 0x7f40d8147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 -- 172.21.15.88:0/4073450728 learned_addr learned my addr 172.21.15.88:0/4073450728 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40d8159190 con 0x7f40d80c7af0 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f40d8159480 con 0x7f40d80c2e60 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f40d806f1f0 con 0x7f40d80c91d0 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d80c2e60 0x7f40d814d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40de221640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80c91d0 0x7f40d8147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80c91d0 msgr2=0x7f40d8147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80c91d0 0x7f40d8147fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d80c2e60 msgr2=0x7f40d814d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d80c2e60 0x7f40d814d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d81484f0 con 0x7f40d80c7af0 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.781+0000 7f40dd21f640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c7af0 0x7f40d8147a70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009ae0 tx=0x7f40c800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.782+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8007b30 con 0x7f40d80c7af0 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.782+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d814c840 con 0x7f40d80c7af0 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.782+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40c8007cd0 con 0x7f40d80c7af0 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.782+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8018650 con 0x7f40d80c7af0 2024-09-21T02:05:04.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.782+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d814cae0 con 0x7f40d80c7af0 2024-09-21T02:05:04.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.783+0000 7f40dfc84640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.783+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f40d815b670 con 0x7f40d815af00 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.783+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.783+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.783+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.783+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.783+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.784+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f40c8021400 con 0x7f40d80c7af0 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.785+0000 7f40ce7fc640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f40bc078b90 0x7f40bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.785+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f40c809efe0 con 0x7f40d80c7af0 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.785+0000 7f40de221640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f40bc078b90 0x7f40bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.785+0000 7f40de221640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f40bc078b90 0x7f40bc07b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40d0035000 tx=0x7f40d0004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:04.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.984+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:04.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.984+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-21T02:05:04.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.984+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:04.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.984+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:04.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:04.984+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T02:05:05.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:05.385+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:05.385+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-21T02:05:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:05.385+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:05.385+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:05.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:05.385+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-21T02:05:06.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:06.187+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:06.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:06.187+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-21T02:05:06.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:06.187+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:06.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:06.187+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:06.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:06.187+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-21T02:05:07.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:07.789+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:07.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:07.789+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-21T02:05:07.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:07.789+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:07.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:07.789+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:07.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:07.789+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-21T02:05:08.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.781+0000 7f40cd7fa640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:05:08.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.781+0000 7f40cd7fa640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.781+0000 7f40cd7fa640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.781+0000 7f40cd7fa640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 msgr2=0x7f40d80cc440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:08.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.781+0000 7f40cd7fa640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d815af00 0x7f40d80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.781+0000 7f40cd7fa640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40060d0 con 0x7f40b4003540 2024-09-21T02:05:08.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.781+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.782+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 msgr2=0x7f40b4005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-21T02:05:08.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.782+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 msgr2=0x7f40b4005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:05:08.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.782+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:05:08.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.782+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:08.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.983+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.984+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40b4006280 con 0x7f40b4003540 2024-09-21T02:05:08.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.984+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40d4019680 tx=0x7f40d4019b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.984+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4027070 con 0x7f40b4003540 2024-09-21T02:05:08.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.985+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f40d4017b10 con 0x7f40b4003540 2024-09-21T02:05:08.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.985+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 msgr2=0x7f40b4005a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.985+0000 7f40ce7fc640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40d4019680 tx=0x7f40d4019b40 comp rx=0 tx=0).stop 2024-09-21T02:05:08.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.985+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 msgr2=0x7f40b4005a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:08.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.985+0000 7f40ce7fc640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40b4003540 0x7f40b4005a20 secure :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40d4019680 tx=0x7f40d4019b40 comp rx=0 tx=0).stop 2024-09-21T02:05:08.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.996+0000 7f40dfc84640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80cc980 0x7f40d8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:08.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.996+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f40d806eae0 con 0x7f40d80cc980 2024-09-21T02:05:08.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.996+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80cc980 0x7f40d8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:08.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40dda20640 1 -- 172.21.15.88:0/4073450728 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f40d8166020 con 0x7f40d80cc980 2024-09-21T02:05:08.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40dda20640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80cc980 0x7f40d8159fd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40d4019680 tx=0x7f40d4000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:08.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4027040 con 0x7f40d80cc980 2024-09-21T02:05:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4547 (secure 0 0 0) 0x7f40d40292d0 con 0x7f40d80cc980 2024-09-21T02:05:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80cc980 msgr2=0x7f40d8159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40ce7fc640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80cc980 0x7f40d8159fd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40d4019680 tx=0x7f40d4000ed0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40ce7fc640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80cc980 msgr2=0x7f40d8159fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40de221640 1 -- 172.21.15.88:0/4073450728 reap_dead start 2024-09-21T02:05:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.997+0000 7f40ce7fc640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d80cc980 0x7f40d8159fd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:08.999 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 10, 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 5124, 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 4, 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:09.000 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:09.001 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.002 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:09.003 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:09.004 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.005 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.006 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:09.007 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:09.008 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:09.009 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:09.010 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:09.011 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f40bc078b90 msgr2=0x7f40bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f40bc078b90 0x7f40bc07b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40d0035000 tx=0x7f40d0004ab0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c7af0 msgr2=0x7f40d8147a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c7af0 0x7f40d8147a70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009ae0 tx=0x7f40c800ff80 comp rx=0 tx=0).stop 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 shutdown_connections 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f40bc078b90 0x7f40bc07b0a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 --2- 172.21.15.88:0/4073450728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40d80c7af0 0x7f40d8147a70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 >> 172.21.15.88:0/4073450728 conn(0x7f40d80720b0 msgr2=0x7f40d8151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 shutdown_connections 2024-09-21T02:05:09.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:08.998+0000 7f40dfc84640 1 -- 172.21.15.88:0/4073450728 wait complete. 2024-09-21T02:05:09.014 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:05:09.014 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-21T02:05:09.014 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-21T02:05:09.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.176+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1250131073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d8151ec0 msgr2=0x7f50d80b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.176+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1250131073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d8151ec0 0x7f50d80b2d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009860 tx=0x7f50d402a5c0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.176+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1250131073 shutdown_connections 2024-09-21T02:05:09.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.176+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1250131073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d8151ec0 0x7f50d80b2d90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.176+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1250131073 >> 172.21.15.88:0/1250131073 conn(0x7f50d8070c30 msgr2=0x7f50d8071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.176+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1250131073 shutdown_connections 2024-09-21T02:05:09.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.176+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1250131073 wait complete. 2024-09-21T02:05:09.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.179+0000 7f50dfa8b640 1 Processor -- start 2024-09-21T02:05:09.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.179+0000 7f50dfa8b640 1 -- start start 2024-09-21T02:05:09.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dfa8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d806cd60 0x7f50d8148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dfa8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80c7d90 0x7f50d80ab450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dfa8b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50d8151ec0 0x7f50d80ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dfa8b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f50d80735f0 con 0x7f50d8151ec0 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dfa8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f50d80c7870 con 0x7f50d80c7d90 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dfa8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f50d8073770 con 0x7f50d806cd60 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50de028640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50d8151ec0 0x7f50d80ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50de028640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50d8151ec0 0x7f50d80ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34252/0 (socket says 172.21.15.88:34252) 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50de028640 1 -- 172.21.15.88:0/1431417417 learned_addr learned my addr 172.21.15.88:0/1431417417 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dd827640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d806cd60 0x7f50d8148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.181+0000 7f50dd026640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80c7d90 0x7f50d80ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50de028640 1 -- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d806cd60 msgr2=0x7f50d8148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50de028640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d806cd60 0x7f50d8148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50de028640 1 -- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80c7d90 msgr2=0x7f50d80ab450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50de028640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80c7d90 0x7f50d80ab450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50de028640 1 -- 172.21.15.88:0/1431417417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d80c61a0 con 0x7f50d8151ec0 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50de028640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50d8151ec0 0x7f50d80ab990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50d000cbc0 tx=0x7f50d000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50ce7fc640 1 -- 172.21.15.88:0/1431417417 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0016020 con 0x7f50d8151ec0 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50ce7fc640 1 -- 172.21.15.88:0/1431417417 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50d000ae30 con 0x7f50d8151ec0 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50ce7fc640 1 -- 172.21.15.88:0/1431417417 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0012660 con 0x7f50d8151ec0 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d81f3450 con 0x7f50d8151ec0 2024-09-21T02:05:09.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.182+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d81f3a10 con 0x7f50d8151ec0 2024-09-21T02:05:09.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.184+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d81502b0 con 0x7f50d8151ec0 2024-09-21T02:05:09.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.184+0000 7f50ce7fc640 1 -- 172.21.15.88:0/1431417417 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f50d0019020 con 0x7f50d8151ec0 2024-09-21T02:05:09.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.185+0000 7f50ce7fc640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f50bc078be0 0x7f50bc07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:09.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.185+0000 7f50ce7fc640 1 -- 172.21.15.88:0/1431417417 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f50d009b360 con 0x7f50d8151ec0 2024-09-21T02:05:09.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.185+0000 7f50dd827640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f50bc078be0 0x7f50bc07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:09.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.186+0000 7f50dd827640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f50bc078be0 0x7f50bc07b0f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:09.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.190+0000 7f50ce7fc640 1 -- 172.21.15.88:0/1431417417 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50d81502b0 con 0x7f50d8151ec0 2024-09-21T02:05:09.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.450+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f50d81f3260 con 0x7f50d8151ec0 2024-09-21T02:05:09.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.451+0000 7f50ce7fc640 1 -- 172.21.15.88:0/1431417417 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f50d0020020 con 0x7f50d8151ec0 2024-09-21T02:05:09.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f50bc078be0 msgr2=0x7f50bc07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f50bc078be0 0x7f50bc07b0f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d4004730 comp rx=0 tx=0).stop 2024-09-21T02:05:09.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50d8151ec0 msgr2=0x7f50d80ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:09.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50d8151ec0 0x7f50d80ab990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50d000cbc0 tx=0x7f50d000a430 comp rx=0 tx=0).stop 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 shutdown_connections 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f50bc078be0 0x7f50bc07b0f0 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d4004730 comp rx=0 tx=0).stop 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50d8151ec0 0x7f50d80ab990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80c7d90 0x7f50d80ab450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 --2- 172.21.15.88:0/1431417417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50d806cd60 0x7f50d8148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 >> 172.21.15.88:0/1431417417 conn(0x7f50d8070c30 msgr2=0x7f50d8076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 shutdown_connections 2024-09-21T02:05:09.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:09.454+0000 7f50dfa8b640 1 -- 172.21.15.88:0/1431417417 wait complete. 2024-09-21T02:05:09.467 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-21T02:05:19.468 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-21T02:05:19.469 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:05:19.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.593+0000 7fa773294640 1 -- 172.21.15.88:0/862229529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cb170 msgr2=0x7fa76c0cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:19.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.593+0000 7fa773294640 1 --2- 172.21.15.88:0/862229529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cb170 0x7fa76c0cbbd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa764005fd0 tx=0x7fa76402ab70 comp rx=0 tx=0).stop 2024-09-21T02:05:19.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.593+0000 7fa773294640 1 -- 172.21.15.88:0/862229529 shutdown_connections 2024-09-21T02:05:19.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.593+0000 7fa773294640 1 --2- 172.21.15.88:0/862229529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cb170 0x7fa76c0cbbd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.593+0000 7fa773294640 1 --2- 172.21.15.88:0/862229529 >> v2:172.21.15.195:3300/0 conn(0x7fa76c0ccd00 0x7fa76c15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.593+0000 7fa773294640 1 -- 172.21.15.88:0/862229529 >> 172.21.15.88:0/862229529 conn(0x7fa76c0cd9f0 msgr2=0x7fa76c0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:19.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.594+0000 7fa773294640 1 -- 172.21.15.88:0/862229529 shutdown_connections 2024-09-21T02:05:19.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.594+0000 7fa773294640 1 -- 172.21.15.88:0/862229529 wait complete. 2024-09-21T02:05:19.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.596+0000 7fa773294640 1 Processor -- start 2024-09-21T02:05:19.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.596+0000 7fa773294640 1 -- start start 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.598+0000 7fa773294640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cad60 0x7fa76c1f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.598+0000 7fa773294640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa76c0cb170 0x7fa76c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa773294640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa76c0ccd00 0x7fa76c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa773294640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c06d470 con 0x7fa76c0cad60 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa773294640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c15c640 con 0x7fa76c0ccd00 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa773294640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c06d5f0 con 0x7fa76c0cb170 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cad60 0x7fa76c1f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:19.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cad60 0x7fa76c1f30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45962/0 (socket says 172.21.15.88:45962) 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 -- 172.21.15.88:0/1047203734 learned_addr learned my addr 172.21.15.88:0/1047203734 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771831640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa76c0ccd00 0x7fa76c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa77082f640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa76c0cb170 0x7fa76c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 -- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa76c0cb170 msgr2=0x7fa76c15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa76c0cb170 0x7fa76c15dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 -- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa76c0ccd00 msgr2=0x7fa76c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa76c0ccd00 0x7fa76c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 -- 172.21.15.88:0/1047203734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c0c77d0 con 0x7fa76c0cad60 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771831640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa76c0ccd00 0x7fa76c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.599+0000 7fa771030640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cad60 0x7fa76c1f30a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa76800d770 tx=0x7fa76800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.600+0000 7fa761ffb640 1 -- 172.21.15.88:0/1047203734 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768017070 con 0x7fa76c0cad60 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.600+0000 7fa761ffb640 1 -- 172.21.15.88:0/1047203734 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa768009d20 con 0x7fa76c0cad60 2024-09-21T02:05:19.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.600+0000 7fa761ffb640 1 -- 172.21.15.88:0/1047203734 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768004280 con 0x7fa76c0cad60 2024-09-21T02:05:19.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.601+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c1561f0 con 0x7fa76c0cad60 2024-09-21T02:05:19.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.601+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c1567b0 con 0x7fa76c0cad60 2024-09-21T02:05:19.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.602+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76c06ddb0 con 0x7fa76c0cad60 2024-09-21T02:05:19.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.603+0000 7fa761ffb640 1 -- 172.21.15.88:0/1047203734 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa768005020 con 0x7fa76c0cad60 2024-09-21T02:05:19.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.603+0000 7fa761ffb640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa750078bc0 0x7fa75007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:19.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.604+0000 7fa761ffb640 1 -- 172.21.15.88:0/1047203734 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa76809b070 con 0x7fa76c0cad60 2024-09-21T02:05:19.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.604+0000 7fa77082f640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa750078bc0 0x7fa75007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:19.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.604+0000 7fa77082f640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa750078bc0 0x7fa75007b0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa76c1ef800 tx=0x7fa75c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:19.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.613+0000 7fa761ffb640 1 -- 172.21.15.88:0/1047203734 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa768020020 con 0x7fa76c0cad60 2024-09-21T02:05:19.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.876+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa76c155f30 con 0x7fa76c0cad60 2024-09-21T02:05:19.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.877+0000 7fa761ffb640 1 -- 172.21.15.88:0/1047203734 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fa768062df0 con 0x7fa76c0cad60 2024-09-21T02:05:19.878 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:05:19.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa750078bc0 msgr2=0x7fa75007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:19.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa750078bc0 0x7fa75007b0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa76c1ef800 tx=0x7fa75c009040 comp rx=0 tx=0).stop 2024-09-21T02:05:19.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cad60 msgr2=0x7fa76c1f30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:19.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cad60 0x7fa76c1f30a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa76800d770 tx=0x7fa76800dc30 comp rx=0 tx=0).stop 2024-09-21T02:05:19.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 shutdown_connections 2024-09-21T02:05:19.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa750078bc0 0x7fa75007b0d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa76c0ccd00 0x7fa76c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa76c0cb170 0x7fa76c15dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 --2- 172.21.15.88:0/1047203734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76c0cad60 0x7fa76c1f30a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:19.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 >> 172.21.15.88:0/1047203734 conn(0x7fa76c0cd9f0 msgr2=0x7fa76c0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:19.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 shutdown_connections 2024-09-21T02:05:19.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:19.880+0000 7fa773294640 1 -- 172.21.15.88:0/1047203734 wait complete. 2024-09-21T02:05:19.896 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-21T02:05:19.896 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-21T02:05:19.896 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-21T02:05:19.896 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:05:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 -- 172.21.15.88:0/466711602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688415b980 msgr2=0x7f68840b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 --2- 172.21.15.88:0/466711602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688415b980 0x7f68840b4c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6880009900 tx=0x7f688002a610 comp rx=0 tx=0).stop 2024-09-21T02:05:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 -- 172.21.15.88:0/466711602 shutdown_connections 2024-09-21T02:05:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 --2- 172.21.15.88:0/466711602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68840cafc0 0x7f688406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 --2- 172.21.15.88:0/466711602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688415b980 0x7f68840b4c10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 -- 172.21.15.88:0/466711602 >> 172.21.15.88:0/466711602 conn(0x7f68840cda90 msgr2=0x7f68840cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 -- 172.21.15.88:0/466711602 shutdown_connections 2024-09-21T02:05:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.010+0000 7f688b2cc640 1 -- 172.21.15.88:0/466711602 wait complete. 2024-09-21T02:05:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.012+0000 7f688b2cc640 1 Processor -- start 2024-09-21T02:05:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.012+0000 7f688b2cc640 1 -- start start 2024-09-21T02:05:20.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.013+0000 7f688b2cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68840cafc0 0x7f6884153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.013+0000 7f688b2cc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840cb990 0x7f6884158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.013+0000 7f688b2cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f688415b980 0x7f68841f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.013+0000 7f688b2cc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68840b4410 con 0x7f68840cb990 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.013+0000 7f688b2cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68840b4130 con 0x7f68840cafc0 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.013+0000 7f688b2cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f68840b46f0 con 0x7f688415b980 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.013+0000 7f6888867640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840cb990 0x7f6884158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840cb990 0x7f6884158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45982/0 (socket says 172.21.15.88:45982) 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 -- 172.21.15.88:0/3389804927 learned_addr learned my addr 172.21.15.88:0/3389804927 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6889068640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68840cafc0 0x7f6884153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6889869640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f688415b980 0x7f68841f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f688415b980 msgr2=0x7f68841f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f688415b980 0x7f68841f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68840cafc0 msgr2=0x7f6884153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68840cafc0 0x7f6884153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 -- 172.21.15.88:0/3389804927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68841f3900 con 0x7f68840cb990 2024-09-21T02:05:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6888867640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840cb990 0x7f6884158930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f687400c3a0 tx=0x7f687400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874017070 con 0x7f68840cb990 2024-09-21T02:05:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68740053c0 con 0x7f68840cb990 2024-09-21T02:05:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6884159350 con 0x7f68840cb990 2024-09-21T02:05:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.014+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874003850 con 0x7f68840cb990 2024-09-21T02:05:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.015+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688415b660 con 0x7f68840cb990 2024-09-21T02:05:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.016+0000 7f688b2cc640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6884157290 0x7f68841576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.016+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68841f3dd0 con 0x7f6884157290 2024-09-21T02:05:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.016+0000 7f6889068640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6884157290 0x7f68841576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.017+0000 7f6889068640 1 -- 172.21.15.88:0/3389804927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6884156e20 con 0x7f6884157290 2024-09-21T02:05:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.017+0000 7f6889068640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6884157290 0x7f68841576a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f688406da20 tx=0x7f6880004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.017+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f687401a070 con 0x7f68840cb990 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6868078b90 0x7f686807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f687409a0e0 con 0x7f68840cb990 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880040070 con 0x7f6884157290 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6880003a00 con 0x7f6884157290 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6884157290 msgr2=0x7f68841576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6884157290 0x7f68841576a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f688406da20 tx=0x7f6880004390 comp rx=0 tx=0).stop 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6884157290 msgr2=0x7f68841576a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6879ffb640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6884157290 0x7f68841576a0 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f688406da20 tx=0x7f6880004390 comp rx=0 tx=0).stop 2024-09-21T02:05:20.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.018+0000 7f6889869640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6868078b90 0x7f686807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.019+0000 7f6889869640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6868078b90 0x7f686807b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f688415a4b0 tx=0x7f687c009560 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.024+0000 7f688b2cc640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68841562f0 0x7f6884156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.024+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68840b4c10 con 0x7f68841562f0 2024-09-21T02:05:20.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.024+0000 7f6889068640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68841562f0 0x7f6884156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6889068640 1 -- 172.21.15.88:0/3389804927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f68841e7c20 con 0x7f68841562f0 2024-09-21T02:05:20.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6889068640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68841562f0 0x7f6884156700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6880009850 tx=0x7f6880009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880035040 con 0x7f68841562f0 2024-09-21T02:05:20.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5053 (secure 0 0 0) 0x7f6880044440 con 0x7f68841562f0 2024-09-21T02:05:20.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68841562f0 msgr2=0x7f6884156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6879ffb640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68841562f0 0x7f6884156700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6880009850 tx=0x7f6880009260 comp rx=0 tx=0).stop 2024-09-21T02:05:20.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6879ffb640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68841562f0 msgr2=0x7f6884156700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:20.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.025+0000 7f6879ffb640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68841562f0 0x7f6884156700 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6880009850 tx=0x7f6880009260 comp rx=0 tx=0).stop 2024-09-21T02:05:20.028 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:20.028 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:20.028 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:20.028 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:05:20.028 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 10, 2024-09-21T02:05:20.028 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 107741, 2024-09-21T02:05:20.028 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 15, 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.029 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:20.031 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:20.033 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.034 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:20.035 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.036 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:20.037 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.038 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:05:20.039 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.040 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:20.041 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6868078b90 msgr2=0x7f686807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6868078b90 0x7f686807b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f688415a4b0 tx=0x7f687c009560 comp rx=0 tx=0).stop 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840cb990 msgr2=0x7f6884158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840cb990 0x7f6884158930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f687400c3a0 tx=0x7f687400c860 comp rx=0 tx=0).stop 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f6889869640 1 -- 172.21.15.88:0/3389804927 reap_dead start 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 shutdown_connections 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 --2- 172.21.15.88:0/3389804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840cb990 0x7f6884158930 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 >> 172.21.15.88:0/3389804927 conn(0x7f68840cda90 msgr2=0x7f6884155690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 shutdown_connections 2024-09-21T02:05:20.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.026+0000 7f688b2cc640 1 -- 172.21.15.88:0/3389804927 wait complete. 2024-09-21T02:05:20.044 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:05:20.044 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-21T02:05:20.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.157+0000 7f8b1c78e640 1 -- 172.21.15.88:0/2314249914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 msgr2=0x7f8b1406cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.157+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/2314249914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b1406cd60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8b1406da20 tx=0x7f8b0c02aa30 comp rx=0 tx=0).stop 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.157+0000 7f8b1c78e640 1 -- 172.21.15.88:0/2314249914 shutdown_connections 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.157+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/2314249914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b1406cd60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.157+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/2314249914 >> v2:172.21.15.195:3300/0 conn(0x7f8b140b4c10 0x7f8b140cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.157+0000 7f8b1c78e640 1 -- 172.21.15.88:0/2314249914 >> 172.21.15.88:0/2314249914 conn(0x7f8b140cda90 msgr2=0x7f8b140cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.157+0000 7f8b1c78e640 1 -- 172.21.15.88:0/2314249914 shutdown_connections 2024-09-21T02:05:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.158+0000 7f8b1c78e640 1 -- 172.21.15.88:0/2314249914 wait complete. 2024-09-21T02:05:20.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.160+0000 7f8b1c78e640 1 Processor -- start 2024-09-21T02:05:20.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.160+0000 7f8b1c78e640 1 -- start start 2024-09-21T02:05:20.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1c78e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b140b4c10 0x7f8b14156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1c78e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b14154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1c78e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1415b980 0x7f8b141e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1c78e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b140b4410 con 0x7f8b140cafc0 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1c78e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b140b4130 con 0x7f8b1415b980 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1c78e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8b140b46f0 con 0x7f8b140b4c10 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b19d29640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b14154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b19d29640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b14154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46020/0 (socket says 172.21.15.88:46020) 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b19d29640 1 -- 172.21.15.88:0/1280199613 learned_addr learned my addr 172.21.15.88:0/1280199613 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1a52a640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b140b4c10 0x7f8b14156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.162+0000 7f8b1ad2b640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1415b980 0x7f8b141e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b19d29640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b140b4c10 msgr2=0x7f8b14156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b19d29640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b140b4c10 0x7f8b14156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b19d29640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1415b980 msgr2=0x7f8b141e1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b19d29640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b1415b980 0x7f8b141e1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b19d29640 1 -- 172.21.15.88:0/1280199613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b141570f0 con 0x7f8b140cafc0 2024-09-21T02:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b19d29640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b14154210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400e790 tx=0x7f8b0400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400eea0 con 0x7f8b140cafc0 2024-09-21T02:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b141e2590 con 0x7f8b140cafc0 2024-09-21T02:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b0400ccc0 con 0x7f8b140cafc0 2024-09-21T02:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04005280 con 0x7f8b140cafc0 2024-09-21T02:05:20.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.163+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b141f2940 con 0x7f8b140cafc0 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.164+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b141e1fa0 0x7f8b14157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.164+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14156720 con 0x7f8b141e1fa0 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.165+0000 7f8b1a52a640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b141e1fa0 0x7f8b14157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.165+0000 7f8b1a52a640 1 -- 172.21.15.88:0/1280199613 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b140b5950 con 0x7f8b141e1fa0 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.165+0000 7f8b1a52a640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b141e1fa0 0x7f8b14157d50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000f570 tx=0x7f8b1000fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.166+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100052e0 con 0x7f8b141e1fa0 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.166+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8b10018070 con 0x7f8b141e1fa0 2024-09-21T02:05:20.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.166+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b141e1fa0 msgr2=0x7f8b14157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.166+0000 7f8b0affd640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b141e1fa0 0x7f8b14157d50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000f570 tx=0x7f8b1000fa30 comp rx=0 tx=0).stop 2024-09-21T02:05:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.166+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b141e1fa0 msgr2=0x7f8b14157d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.166+0000 7f8b0affd640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b141e1fa0 0x7f8b14157d50 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000f570 tx=0x7f8b1000fa30 comp rx=0 tx=0).stop 2024-09-21T02:05:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.166+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b04005420 con 0x7f8b140cafc0 2024-09-21T02:05:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.167+0000 7f8b0affd640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8af8078670 0x7f8af807ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.167+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8b0409b6a0 con 0x7f8b140cafc0 2024-09-21T02:05:20.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.167+0000 7f8b1a52a640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8af8078670 0x7f8af807ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.167+0000 7f8b1a52a640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8af8078670 0x7f8af807ab80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000fa70 tx=0x7f8b10005000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.178+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b14158290 0x7f8b141586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.178+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b140c4920 con 0x7f8b14158290 2024-09-21T02:05:20.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.178+0000 7f8b1ad2b640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b14158290 0x7f8b141586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b1ad2b640 1 -- 172.21.15.88:0/1280199613 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8b140cbae0 con 0x7f8b14158290 2024-09-21T02:05:20.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b1ad2b640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b14158290 0x7f8b141586a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b1406eb10 tx=0x7f8b0c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c033070 con 0x7f8b14158290 2024-09-21T02:05:20.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5027 (secure 0 0 0) 0x7f8b0c004240 con 0x7f8b14158290 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b14158290 msgr2=0x7f8b141586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b0affd640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b14158290 0x7f8b141586a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b1406eb10 tx=0x7f8b0c003ba0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b0affd640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b14158290 msgr2=0x7f8b141586a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.179+0000 7f8b0affd640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b14158290 0x7f8b141586a0 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b1406eb10 tx=0x7f8b0c003ba0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:05:20.181 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 10, 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 107852, 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 16, 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.182 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.183 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:20.184 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:20.185 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:20.186 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.187 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:20.188 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.189 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.190 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:20.191 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.192 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.193 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.194 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.195 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:20.195 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8af8078670 msgr2=0x7f8af807ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8af8078670 0x7f8af807ab80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000fa70 tx=0x7f8b10005000 comp rx=0 tx=0).stop 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1ad2b640 1 -- 172.21.15.88:0/1280199613 reap_dead start 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 msgr2=0x7f8b14154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b14154210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400e790 tx=0x7f8b0400ec50 comp rx=0 tx=0).stop 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 shutdown_connections 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 --2- 172.21.15.88:0/1280199613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b140cafc0 0x7f8b14154210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 >> 172.21.15.88:0/1280199613 conn(0x7f8b140cda90 msgr2=0x7f8b14155320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 shutdown_connections 2024-09-21T02:05:20.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.180+0000 7f8b1c78e640 1 -- 172.21.15.88:0/1280199613 wait complete. 2024-09-21T02:05:20.197 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:05:20.197 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-21T02:05:20.197 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-21T02:05:20.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.311+0000 7f4f754cc640 1 -- 172.21.15.88:0/2113499358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7015ddb0 msgr2=0x7f4f7006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.311+0000 7f4f754cc640 1 --2- 172.21.15.88:0/2113499358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7015ddb0 0x7f4f7006d170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4f68005fd0 tx=0x7f4f6802a8b0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.311+0000 7f4f754cc640 1 -- 172.21.15.88:0/2113499358 shutdown_connections 2024-09-21T02:05:20.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.311+0000 7f4f754cc640 1 --2- 172.21.15.88:0/2113499358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7015ddb0 0x7f4f7006d170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.311+0000 7f4f754cc640 1 -- 172.21.15.88:0/2113499358 >> 172.21.15.88:0/2113499358 conn(0x7f4f700cda70 msgr2=0x7f4f700cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.311+0000 7f4f754cc640 1 -- 172.21.15.88:0/2113499358 shutdown_connections 2024-09-21T02:05:20.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.311+0000 7f4f754cc640 1 -- 172.21.15.88:0/2113499358 wait complete. 2024-09-21T02:05:20.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.313+0000 7f4f754cc640 1 Processor -- start 2024-09-21T02:05:20.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.313+0000 7f4f754cc640 1 -- start start 2024-09-21T02:05:20.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f754cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7006cd60 0x7f4f701523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f754cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f700b5bc0 0x7f4f700c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f754cc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f7015ddb0 0x7f4f7014c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f754cc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f700cd0d0 con 0x7f4f7015ddb0 2024-09-21T02:05:20.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f754cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7015d180 con 0x7f4f7006cd60 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f754cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7015bb70 con 0x7f4f700b5bc0 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f6f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f7015ddb0 0x7f4f7014c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f6f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f7015ddb0 0x7f4f7014c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46030/0 (socket says 172.21.15.88:46030) 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f6f7fe640 1 -- 172.21.15.88:0/361891278 learned_addr learned my addr 172.21.15.88:0/361891278 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f6e7fc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f700b5bc0 0x7f4f700c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.314+0000 7f4f6effd640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7006cd60 0x7f4f701523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f6f7fe640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f700b5bc0 msgr2=0x7f4f700c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f6f7fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f700b5bc0 0x7f4f700c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f6f7fe640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7006cd60 msgr2=0x7f4f701523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f6f7fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f7006cd60 0x7f4f701523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f6f7fe640 1 -- 172.21.15.88:0/361891278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700c7900 con 0x7f4f7015ddb0 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f6e7fc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f700b5bc0 0x7f4f700c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:05:20.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f6f7fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f7015ddb0 0x7f4f7014c550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4f6802aea0 tx=0x7f4f680316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68031990 con 0x7f4f7015ddb0 2024-09-21T02:05:20.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f700c7050 con 0x7f4f7015ddb0 2024-09-21T02:05:20.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f700c7620 con 0x7f4f7015ddb0 2024-09-21T02:05:20.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f68031c40 con 0x7f4f7015ddb0 2024-09-21T02:05:20.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.315+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68033330 con 0x7f4f7015ddb0 2024-09-21T02:05:20.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.316+0000 7f4f754cc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50002a70 0x7f4f50004f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.317+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4f50007f10 con 0x7f4f50002a70 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.317+0000 7f4f6effd640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50002a70 0x7f4f50004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.317+0000 7f4f6effd640 1 -- 172.21.15.88:0/361891278 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f50008110 con 0x7f4f50002a70 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.317+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f680334d0 con 0x7f4f7015ddb0 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.317+0000 7f4f6effd640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50002a70 0x7f4f50004f80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000e3c0 tx=0x7f4f6000e880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4f680bc180 con 0x7f4f7015ddb0 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f6e7fc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60019070 con 0x7f4f50002a70 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4f600052e0 con 0x7f4f50002a70 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50002a70 msgr2=0x7f4f50004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50002a70 0x7f4f50004f80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000e3c0 tx=0x7f4f6000e880 comp rx=0 tx=0).stop 2024-09-21T02:05:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50002a70 msgr2=0x7f4f50004f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:20.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.318+0000 7f4f577fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50002a70 0x7f4f50004f80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.319+0000 7f4f6e7fc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f70152360 tx=0x7f4f64009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.324+0000 7f4f754cc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50006f80 0x7f4f50007460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.324+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5000a4f0 con 0x7f4f50006f80 2024-09-21T02:05:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.324+0000 7f4f6effd640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50006f80 0x7f4f50007460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.325+0000 7f4f6effd640 1 -- 172.21.15.88:0/361891278 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4f500079a0 con 0x7f4f50006f80 2024-09-21T02:05:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.325+0000 7f4f6effd640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50006f80 0x7f4f50007460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000e3c0 tx=0x7f4f60000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.325+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60019040 con 0x7f4f50006f80 2024-09-21T02:05:20.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4549 (secure 0 0 0) 0x7f4f600035f0 con 0x7f4f50006f80 2024-09-21T02:05:20.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50006f80 msgr2=0x7f4f50007460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f577fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50006f80 0x7f4f50007460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000e3c0 tx=0x7f4f60000ed0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f577fe640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50006f80 msgr2=0x7f4f50007460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:20.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f577fe640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f50006f80 0x7f4f50007460 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000e3c0 tx=0x7f4f60000ed0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.327 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:20.327 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 10, 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 16453, 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 16, 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:20.328 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:20.329 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:20.330 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:20.331 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.333 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:20.334 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.335 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.336 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.337 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:20.338 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:20.339 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:20.339 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:20.339 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:20.339 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:20.339 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:20.339 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:20.339 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:20.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4f5c078bc0 msgr2=0x7f4f5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4f5c078bc0 0x7f4f5c07b0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f70152360 tx=0x7f4f64009040 comp rx=0 tx=0).stop 2024-09-21T02:05:20.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f6f7fe640 1 -- 172.21.15.88:0/361891278 reap_dead start 2024-09-21T02:05:20.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f7015ddb0 msgr2=0x7f4f7014c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f7015ddb0 0x7f4f7014c550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4f6802aea0 tx=0x7f4f680316f0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 shutdown_connections 2024-09-21T02:05:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 --2- 172.21.15.88:0/361891278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f7015ddb0 0x7f4f7014c550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 >> 172.21.15.88:0/361891278 conn(0x7f4f700cda70 msgr2=0x7f4f701517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 shutdown_connections 2024-09-21T02:05:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.326+0000 7f4f754cc640 1 -- 172.21.15.88:0/361891278 wait complete. 2024-09-21T02:05:20.341 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:05:20.341 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-21T02:05:20.342 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-21T02:05:20.342 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-21T02:05:20.342 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-21T02:05:20.342 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T02:05:20.342 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:05:20.354 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T02:05:20.354 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-21T02:05:20.355 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:05:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.467+0000 7fca1ba4d640 1 -- 172.21.15.88:0/2921787207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca000039e0 msgr2=0x7fca00003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.467+0000 7fca1ba4d640 1 --2- 172.21.15.88:0/2921787207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca000039e0 0x7fca00003e10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fca10001740 tx=0x7fca00005830 comp rx=0 tx=0).stop 2024-09-21T02:05:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.467+0000 7fca19fea640 1 -- 172.21.15.88:0/2921787207 reap_dead start 2024-09-21T02:05:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.467+0000 7fca1ba4d640 1 -- 172.21.15.88:0/2921787207 shutdown_connections 2024-09-21T02:05:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.467+0000 7fca1ba4d640 1 -- 172.21.15.88:0/2921787207 >> 172.21.15.88:0/2921787207 conn(0x7fca140ca8d0 msgr2=0x7fca140cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.467+0000 7fca1ba4d640 1 -- 172.21.15.88:0/2921787207 shutdown_connections 2024-09-21T02:05:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.467+0000 7fca1ba4d640 1 -- 172.21.15.88:0/2921787207 wait complete. 2024-09-21T02:05:20.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.469+0000 7fca1ba4d640 1 Processor -- start 2024-09-21T02:05:20.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.469+0000 7fca1ba4d640 1 -- start start 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca1ba4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca1414bab0 0x7fca141e1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca1ba4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca141e1e90 0x7fca141f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca1ba4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca141f3520 0x7fca141e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca1ba4d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca14158a40 con 0x7fca141e1e90 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca1ba4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fca140c4dd0 con 0x7fca141f3520 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca1ba4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fca140cb770 con 0x7fca1414bab0 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca141e1e90 0x7fca141f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca197e9640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca1414bab0 msgr2=0x7fca141e1950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca197e9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca1414bab0 0x7fca141e1950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca141e1e90 0x7fca141f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46038/0 (socket says 172.21.15.88:46038) 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 -- 172.21.15.88:0/3572865780 learned_addr learned my addr 172.21.15.88:0/3572865780 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca19fea640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca141f3520 0x7fca141e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 -- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca1414bab0 msgr2=0x7fca141e1950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca1414bab0 0x7fca141e1950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 -- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca141f3520 msgr2=0x7fca141e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca141f3520 0x7fca141e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 -- 172.21.15.88:0/3572865780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1414e930 con 0x7fca141e1e90 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.471+0000 7fca18fe8640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca141e1e90 0x7fca141f2fe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca04006630 tx=0x7fca04003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.472+0000 7fca09ffb640 1 -- 172.21.15.88:0/3572865780 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04014ca0 con 0x7fca141e1e90 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.472+0000 7fca09ffb640 1 -- 172.21.15.88:0/3572865780 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca04018030 con 0x7fca141e1e90 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.472+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca141e27f0 con 0x7fca141e1e90 2024-09-21T02:05:20.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.472+0000 7fca09ffb640 1 -- 172.21.15.88:0/3572865780 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0401e690 con 0x7fca141e1e90 2024-09-21T02:05:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.472+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca141e2db0 con 0x7fca141e1e90 2024-09-21T02:05:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.473+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca140759e0 con 0x7fca141e1e90 2024-09-21T02:05:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.474+0000 7fca09ffb640 1 -- 172.21.15.88:0/3572865780 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca0400f430 con 0x7fca141e1e90 2024-09-21T02:05:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.475+0000 7fca09ffb640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc9f8078bc0 0x7fc9f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.475+0000 7fca09ffb640 1 -- 172.21.15.88:0/3572865780 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fca0409f680 con 0x7fca141e1e90 2024-09-21T02:05:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.476+0000 7fca197e9640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc9f8078bc0 0x7fc9f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:20.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.476+0000 7fca197e9640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc9f8078bc0 0x7fc9f807b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca10001710 tx=0x7fca10001270 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:20.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.483+0000 7fca09ffb640 1 -- 172.21.15.88:0/3572865780 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fca040675f0 con 0x7fca141e1e90 2024-09-21T02:05:20.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.785+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fca14075040 con 0x7fca141e1e90 2024-09-21T02:05:20.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.786+0000 7fca09ffb640 1 -- 172.21.15.88:0/3572865780 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fca0406e070 con 0x7fca141e1e90 2024-09-21T02:05:20.787 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:05:20.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc9f8078bc0 msgr2=0x7fc9f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc9f8078bc0 0x7fc9f807b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca10001710 tx=0x7fca10001270 comp rx=0 tx=0).stop 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca141e1e90 msgr2=0x7fca141f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca141e1e90 0x7fca141f2fe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca04006630 tx=0x7fca04003aa0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 shutdown_connections 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc9f8078bc0 0x7fc9f807b0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca141f3520 0x7fca141e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca141e1e90 0x7fca141f2fe0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 --2- 172.21.15.88:0/3572865780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca1414bab0 0x7fca141e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.788+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 >> 172.21.15.88:0/3572865780 conn(0x7fca140ca8d0 msgr2=0x7fca14071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.789+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 shutdown_connections 2024-09-21T02:05:20.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:20.789+0000 7fca1ba4d640 1 -- 172.21.15.88:0/3572865780 wait complete. 2024-09-21T02:05:23.805 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 -- 172.21.15.88:0/3663065555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 msgr2=0x7fadb00c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/3663065555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb00c41c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fadb0070eb0 tx=0x7fada402a680 comp rx=0 tx=0).stop 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 -- 172.21.15.88:0/3663065555 shutdown_connections 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/3663065555 >> v2:172.21.15.195:3300/0 conn(0x7fadb0074490 0x7fadb00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/3663065555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb00c41c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 -- 172.21.15.88:0/3663065555 >> 172.21.15.88:0/3663065555 conn(0x7fadb00ca8d0 msgr2=0x7fadb00cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 -- 172.21.15.88:0/3663065555 shutdown_connections 2024-09-21T02:05:23.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.925+0000 7fadb4a4d640 1 -- 172.21.15.88:0/3663065555 wait complete. 2024-09-21T02:05:23.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.928+0000 7fadb4a4d640 1 Processor -- start 2024-09-21T02:05:23.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.928+0000 7fadb4a4d640 1 -- start start 2024-09-21T02:05:23.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.930+0000 7fadb4a4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadb0074490 0x7fadb015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:23.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.930+0000 7fadb4a4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb01f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.930+0000 7fadae59c640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadb0074490 msgr2=0x7fadb015b3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.930+0000 7fadae59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadb0074490 0x7fadb015b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.930+0000 7fadadd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb01f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb01f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46060/0 (socket says 172.21.15.88:46060) 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.930+0000 7fadb4a4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadb01526a0 0x7fadb0145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 -- 172.21.15.88:0/754303600 learned_addr learned my addr 172.21.15.88:0/754303600 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fadb00cb770 con 0x7fadb00c97e0 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0158a40 con 0x7fadb01526a0 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fadb00759e0 con 0x7fadb0074490 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadaed9d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadb01526a0 0x7fadb0145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 -- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadb0074490 msgr2=0x7fadb015b3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadb0074490 0x7fadb015b3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 -- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadb01526a0 msgr2=0x7fadb0145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:23.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadb01526a0 0x7fadb0145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb015aea0 con 0x7fadb00c97e0 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadaed9d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadb01526a0 0x7fadb0145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadadd9b640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb01f2630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fada40099c0 tx=0x7fada40045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fad9effd640 1 -- 172.21.15.88:0/754303600 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4035040 con 0x7fadb00c97e0 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fad9effd640 1 -- 172.21.15.88:0/754303600 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada403e070 con 0x7fadb00c97e0 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fad9effd640 1 -- 172.21.15.88:0/754303600 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4004a00 con 0x7fadb00c97e0 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.931+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb01f32f0 con 0x7fadb00c97e0 2024-09-21T02:05:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.932+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01f3010 con 0x7fadb00c97e0 2024-09-21T02:05:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.933+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb014e7a0 con 0x7fadb00c97e0 2024-09-21T02:05:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.934+0000 7fad9effd640 1 -- 172.21.15.88:0/754303600 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fada4004ba0 con 0x7fadb00c97e0 2024-09-21T02:05:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.934+0000 7fad9effd640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad90078bc0 0x7fad9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.934+0000 7fad9effd640 1 -- 172.21.15.88:0/754303600 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fada40bd810 con 0x7fadb00c97e0 2024-09-21T02:05:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.935+0000 7fadae59c640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad90078bc0 0x7fad9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:23.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.935+0000 7fadae59c640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad90078bc0 0x7fad9007b0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fada0007fd0 tx=0x7fada0007900 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:23.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:23.943+0000 7fad9effd640 1 -- 172.21.15.88:0/754303600 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fadb015aea0 con 0x7fadb00c97e0 2024-09-21T02:05:24.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.201+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fadb015a630 con 0x7fadb00c97e0 2024-09-21T02:05:24.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.202+0000 7fad9effd640 1 -- 172.21.15.88:0/754303600 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fada4085bb0 con 0x7fadb00c97e0 2024-09-21T02:05:24.203 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:05:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad90078bc0 msgr2=0x7fad9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad90078bc0 0x7fad9007b0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fada0007fd0 tx=0x7fada0007900 comp rx=0 tx=0).stop 2024-09-21T02:05:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 msgr2=0x7fadb01f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb01f2630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fada40099c0 tx=0x7fada40045a0 comp rx=0 tx=0).stop 2024-09-21T02:05:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 shutdown_connections 2024-09-21T02:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad90078bc0 0x7fad9007b0d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadb01526a0 0x7fadb0145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadb00c97e0 0x7fadb01f2630 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 --2- 172.21.15.88:0/754303600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadb0074490 0x7fadb015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 >> 172.21.15.88:0/754303600 conn(0x7fadb00ca8d0 msgr2=0x7fadb0151820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 shutdown_connections 2024-09-21T02:05:24.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:24.204+0000 7fadb4a4d640 1 -- 172.21.15.88:0/754303600 wait complete. 2024-09-21T02:05:27.217 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:05:27.578 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:05:27.580 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:05:27.866 INFO:tasks.workunit.client.0.smithi088.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T02:05:27.866 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr: git switch -c 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr:Or undo this operation with: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr: git switch - 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-09-21T02:05:27.867 INFO:tasks.workunit.client.0.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T02:05:27.868 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-09-21T02:05:27.868 INFO:tasks.workunit.client.0.smithi088.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:05:27.871 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T02:05:27.900 INFO:tasks.workunit.client.0.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T02:05:27.902 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T02:05:27.903 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T02:05:28.460 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T02:05:28.532 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T02:05:28.572 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T02:05:28.574 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T02:05:28.574 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T02:05:28.612 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T02:05:28.616 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:05:28.616 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T02:05:28.672 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-21T02:05:28.673 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-21T02:05:28.673 DEBUG:teuthology.orchestra.run.smithi088:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-21T02:05:28.735 INFO:tasks.workunit.client.0.smithi088.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-21T02:05:28.811 INFO:tasks.workunit.client.0.smithi088.stdout: Iozone: Performance Test of File I/O 2024-09-21T02:05:28.811 INFO:tasks.workunit.client.0.smithi088.stdout: Version $Revision: 3.494 $ 2024-09-21T02:05:28.811 INFO:tasks.workunit.client.0.smithi088.stdout: Compiled for 64 bit mode. 2024-09-21T02:05:28.811 INFO:tasks.workunit.client.0.smithi088.stdout: Build: linux-AMD64 2024-09-21T02:05:28.811 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:28.811 INFO:tasks.workunit.client.0.smithi088.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Run began: Sat Sep 21 02:05:28 2024 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Include close in write timing 2024-09-21T02:05:28.812 INFO:tasks.workunit.client.0.smithi088.stdout: Include fsync in write timing 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: File size set to 1048576 kB 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: Record Size 16 kB 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: Output is in kBytes/sec 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: Processor cache line size set to 32 bytes. 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: File stride size set to 17 * record size. 2024-09-21T02:05:28.813 INFO:tasks.workunit.client.0.smithi088.stdout: Throughput test with 1 process 2024-09-21T02:05:28.814 INFO:tasks.workunit.client.0.smithi088.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-21T02:05:30.213 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:30.213 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 initial writers = 749128.38 kB/sec 2024-09-21T02:05:30.213 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 initial writers = 749033.09 kB/sec 2024-09-21T02:05:30.213 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 749128.38 kB/sec 2024-09-21T02:05:30.213 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 749128.38 kB/sec 2024-09-21T02:05:30.213 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 749128.38 kB/sec 2024-09-21T02:05:30.213 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:34.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.918+0000 7fdf574c5640 1 -- 172.21.15.88:0/1425610020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf3c0039b0 msgr2=0x7fdf3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.918+0000 7fdf574c5640 1 --2- 172.21.15.88:0/1425610020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf3c0039b0 0x7fdf3c003e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf34008e50 tx=0x7fdf3c005830 comp rx=0 tx=0).stop 2024-09-21T02:05:34.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.918+0000 7fdf55a62640 1 -- 172.21.15.88:0/1425610020 reap_dead start 2024-09-21T02:05:34.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.918+0000 7fdf574c5640 1 -- 172.21.15.88:0/1425610020 shutdown_connections 2024-09-21T02:05:34.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.918+0000 7fdf574c5640 1 -- 172.21.15.88:0/1425610020 >> 172.21.15.88:0/1425610020 conn(0x7fdf500ca8d0 msgr2=0x7fdf500cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:34.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.918+0000 7fdf574c5640 1 -- 172.21.15.88:0/1425610020 shutdown_connections 2024-09-21T02:05:34.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.918+0000 7fdf574c5640 1 -- 172.21.15.88:0/1425610020 wait complete. 2024-09-21T02:05:34.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.920+0000 7fdf574c5640 1 Processor -- start 2024-09-21T02:05:34.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.920+0000 7fdf574c5640 1 -- start start 2024-09-21T02:05:34.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf574c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50147240 0x7fdf501f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf574c5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf501f2350 0x7fdf5014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf574c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf50147650 0x7fdf50145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf574c5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf500cb770 con 0x7fdf501f2350 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf574c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50158a40 con 0x7fdf50147240 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf574c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fdf500759e0 con 0x7fdf50147650 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf55a62640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf50147650 msgr2=0x7fdf50145480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf501f2350 0x7fdf5014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf55a62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf50147650 0x7fdf50145480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf501f2350 0x7fdf5014ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48266/0 (socket says 172.21.15.88:48266) 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 -- 172.21.15.88:0/4242047120 learned_addr learned my addr 172.21.15.88:0/4242047120 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf55261640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50147240 0x7fdf501f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 -- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf50147650 msgr2=0x7fdf50145480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf50147650 0x7fdf50145480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 -- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50147240 msgr2=0x7fdf501f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50147240 0x7fdf501f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 -- 172.21.15.88:0/4242047120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf500cc120 con 0x7fdf501f2350 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.922+0000 7fdf54a60640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf501f2350 0x7fdf5014ed60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007a30 tx=0x7fdf4c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.923+0000 7fdf41ffb640 1 -- 172.21.15.88:0/4242047120 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c014dd0 con 0x7fdf501f2350 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.923+0000 7fdf41ffb640 1 -- 172.21.15.88:0/4242047120 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf4c003b20 con 0x7fdf501f2350 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.923+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf501504f0 con 0x7fdf501f2350 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.923+0000 7fdf41ffb640 1 -- 172.21.15.88:0/4242047120 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c004020 con 0x7fdf501f2350 2024-09-21T02:05:34.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.923+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5014fa10 con 0x7fdf501f2350 2024-09-21T02:05:34.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.924+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5014f800 con 0x7fdf501f2350 2024-09-21T02:05:34.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.925+0000 7fdf41ffb640 1 -- 172.21.15.88:0/4242047120 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf4c0041c0 con 0x7fdf501f2350 2024-09-21T02:05:34.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.926+0000 7fdf41ffb640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fdf34079f20 0x7fdf3407c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:34.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.926+0000 7fdf41ffb640 1 -- 172.21.15.88:0/4242047120 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fdf4c09db50 con 0x7fdf501f2350 2024-09-21T02:05:34.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.926+0000 7fdf55261640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fdf34079f20 0x7fdf3407c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:34.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.927+0000 7fdf55261640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fdf34079f20 0x7fdf3407c430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf44036000 tx=0x7fdf44015040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:34.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:34.930+0000 7fdf41ffb640 1 -- 172.21.15.88:0/4242047120 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf4c025030 con 0x7fdf501f2350 2024-09-21T02:05:35.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.196+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdf500753f0 con 0x7fdf501f2350 2024-09-21T02:05:35.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.197+0000 7fdf41ffb640 1 -- 172.21.15.88:0/4242047120 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fdf4c01c030 con 0x7fdf501f2350 2024-09-21T02:05:35.199 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":14,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:05:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fdf34079f20 msgr2=0x7fdf3407c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fdf34079f20 0x7fdf3407c430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf44036000 tx=0x7fdf44015040 comp rx=0 tx=0).stop 2024-09-21T02:05:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf501f2350 msgr2=0x7fdf5014ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf501f2350 0x7fdf5014ed60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007a30 tx=0x7fdf4c007ef0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 shutdown_connections 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fdf34079f20 0x7fdf3407c430 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf50147650 0x7fdf50145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf501f2350 0x7fdf5014ed60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 --2- 172.21.15.88:0/4242047120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf50147240 0x7fdf501f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 >> 172.21.15.88:0/4242047120 conn(0x7fdf500ca8d0 msgr2=0x7fdf501517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 shutdown_connections 2024-09-21T02:05:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.200+0000 7fdf574c5640 1 -- 172.21.15.88:0/4242047120 wait complete. 2024-09-21T02:05:35.223 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-21T02:05:35.223 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-21T02:05:35.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:05:35.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.579+0000 7effc0369640 1 -- 172.21.15.88:0/1816713218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 msgr2=0x7effb815c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.579+0000 7effc0369640 1 --2- 172.21.15.88:0/1816713218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb815c040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7effb4005fd0 tx=0x7effb402a4a0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.579+0000 7effc0369640 1 -- 172.21.15.88:0/1816713218 shutdown_connections 2024-09-21T02:05:35.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.579+0000 7effc0369640 1 --2- 172.21.15.88:0/1816713218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb815c040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.579+0000 7effc0369640 1 -- 172.21.15.88:0/1816713218 >> 172.21.15.88:0/1816713218 conn(0x7effb80720b0 msgr2=0x7effb80cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:35.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.580+0000 7effc0369640 1 -- 172.21.15.88:0/1816713218 shutdown_connections 2024-09-21T02:05:35.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.580+0000 7effc0369640 1 -- 172.21.15.88:0/1816713218 wait complete. 2024-09-21T02:05:35.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.583+0000 7effc0369640 1 Processor -- start 2024-09-21T02:05:35.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.583+0000 7effc0369640 1 -- start start 2024-09-21T02:05:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effc0369640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb8157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effc0369640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb815c580 0x7effb815dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effc0369640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effb815f690 0x7effb80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effc0369640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7effb80c8dc0 con 0x7effb8155de0 2024-09-21T02:05:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effc0369640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7effb80c4730 con 0x7effb815c580 2024-09-21T02:05:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effc0369640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7effb80b40d0 con 0x7effb815f690 2024-09-21T02:05:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb8157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb8157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48280/0 (socket says 172.21.15.88:48280) 2024-09-21T02:05:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 -- 172.21.15.88:0/4105247845 learned_addr learned my addr 172.21.15.88:0/4105247845 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe906640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effb815f690 msgr2=0x7effb80cd7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:05:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe906640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effb815f690 0x7effb80cd7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:05:35.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbd904640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb815c580 0x7effb815dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effb815f690 msgr2=0x7effb80cd7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:05:35.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effb815f690 0x7effb80cd7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb815c580 msgr2=0x7effb815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb815c580 0x7effb815dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbe105640 1 -- 172.21.15.88:0/4105247845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb8157d90 con 0x7effb8155de0 2024-09-21T02:05:35.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.586+0000 7effbd904640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effb815c580 0x7effb815dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:05:35.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.587+0000 7effbe105640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb8157850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7effb4034040 tx=0x7effb40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.587+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb403e030 con 0x7effb8155de0 2024-09-21T02:05:35.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.587+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effb40047a0 con 0x7effb8155de0 2024-09-21T02:05:35.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.587+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4035040 con 0x7effb8155de0 2024-09-21T02:05:35.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.587+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb815eb10 con 0x7effb8155de0 2024-09-21T02:05:35.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.588+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb80cd290 con 0x7effb8155de0 2024-09-21T02:05:35.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.590+0000 7effc0369640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8157f60 0x7effb80cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.590+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7effb80cd460 con 0x7effb8157f60 2024-09-21T02:05:35.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.590+0000 7effbd904640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8157f60 0x7effb80cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.590+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effb4041030 con 0x7effb8155de0 2024-09-21T02:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.590+0000 7effbd904640 1 -- 172.21.15.88:0/4105247845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effb80c42f0 con 0x7effb8157f60 2024-09-21T02:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.591+0000 7effbd904640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8157f60 0x7effb80cdd30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effb80cb2e0 tx=0x7effa800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.591+0000 7effaeffd640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7eff9c078b70 0x7eff9c07b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.591+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7effb40bb580 con 0x7effb8155de0 2024-09-21T02:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.591+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8015070 con 0x7effb8157f60 2024-09-21T02:05:35.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.592+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7effa8011800 con 0x7effb8157f60 2024-09-21T02:05:35.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.592+0000 7effbe906640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7eff9c078b70 0x7eff9c07b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.592+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8157f60 msgr2=0x7effb80cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.592+0000 7effaeffd640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8157f60 0x7effb80cdd30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effb80cb2e0 tx=0x7effa800c950 comp rx=0 tx=0).stop 2024-09-21T02:05:35.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.592+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8157f60 msgr2=0x7effb80cdd30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:35.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.592+0000 7effaeffd640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8157f60 0x7effb80cdd30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.595+0000 7effbe906640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7eff9c078b70 0x7eff9c07b080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7effb815b540 tx=0x7effb0007d10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.602+0000 7effc0369640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb815b590 0x7effb814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.602+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7effb815e8c0 con 0x7effb815b590 2024-09-21T02:05:35.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.602+0000 7effbd904640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb815b590 0x7effb814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.602+0000 7effbd904640 1 -- 172.21.15.88:0/4105247845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7effb80c80d0 con 0x7effb815b590 2024-09-21T02:05:35.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.603+0000 7effbd904640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb815b590 0x7effb814c160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effa8017000 tx=0x7effa80046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.603+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8003b00 con 0x7effb815b590 2024-09-21T02:05:35.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.603+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5041 (secure 0 0 0) 0x7effa8003b00 con 0x7effb815b590 2024-09-21T02:05:35.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.603+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb815b590 msgr2=0x7effb814c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.603+0000 7effaeffd640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb815b590 0x7effb814c160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effa8017000 tx=0x7effa80046f0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effaeffd640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb815b590 msgr2=0x7effb814c160 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 14, 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 123319, 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: 1 2024-09-21T02:05:35.607 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:35.608 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:35.609 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:35.610 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:35.611 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:35.612 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.613 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:35.614 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:35.615 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:35.616 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:35.617 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:05:35.618 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:05:35.619 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.620 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.620 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.620 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:35.620 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effaeffd640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb815b590 0x7effb814c160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7eff9c078b70 msgr2=0x7eff9c07b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effc0369640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7eff9c078b70 0x7eff9c07b080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7effb815b540 tx=0x7effb0007d10 comp rx=0 tx=0).stop 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effbe906640 1 -- 172.21.15.88:0/4105247845 reap_dead start 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 msgr2=0x7effb8157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effc0369640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb8157850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7effb4034040 tx=0x7effb40039e0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.604+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 shutdown_connections 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.605+0000 7effc0369640 1 --2- 172.21.15.88:0/4105247845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effb8155de0 0x7effb8157850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.605+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 >> 172.21.15.88:0/4105247845 conn(0x7effb80720b0 msgr2=0x7effb8074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.605+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 shutdown_connections 2024-09-21T02:05:35.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.605+0000 7effc0369640 1 -- 172.21.15.88:0/4105247845 wait complete. 2024-09-21T02:05:35.623 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-21T02:05:35.623 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-21T02:05:35.624 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:05:35.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.797+0000 7fb4f2f57640 1 -- 172.21.15.88:0/1114089598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4dc003a00 msgr2=0x7fb4dc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.797+0000 7fb4f2f57640 1 --2- 172.21.15.88:0/1114089598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4dc003a00 0x7fb4dc003e50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099d0 tx=0x7fb4dc005830 comp rx=0 tx=0).stop 2024-09-21T02:05:35.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.798+0000 7fb4f2f57640 1 -- 172.21.15.88:0/1114089598 shutdown_connections 2024-09-21T02:05:35.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.798+0000 7fb4f2f57640 1 --2- 172.21.15.88:0/1114089598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4dc003a00 0x7fb4dc003e50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.798+0000 7fb4f2f57640 1 -- 172.21.15.88:0/1114089598 >> 172.21.15.88:0/1114089598 conn(0x7fb4ec071d80 msgr2=0x7fb4ec072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:35.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.798+0000 7fb4f2f57640 1 -- 172.21.15.88:0/1114089598 shutdown_connections 2024-09-21T02:05:35.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.798+0000 7fb4f2f57640 1 -- 172.21.15.88:0/1114089598 wait complete. 2024-09-21T02:05:35.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.804+0000 7fb4f2f57640 1 Processor -- start 2024-09-21T02:05:35.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.806+0000 7fb4f2f57640 1 -- start start 2024-09-21T02:05:35.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.809+0000 7fb4f2f57640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4dc003a00 0x7fb4ec151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.810+0000 7fb4f2f57640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4ec14dcf0 0x7fb4ec1464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.810+0000 7fb4f2f57640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4ec14e100 0x7fb4ec146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.810+0000 7fb4f2f57640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec06ee60 con 0x7fb4ec14e100 2024-09-21T02:05:35.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.811+0000 7fb4f2f57640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec158880 con 0x7fb4dc003a00 2024-09-21T02:05:35.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.811+0000 7fb4f14f4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4ec14e100 0x7fb4ec146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.811+0000 7fb4f14f4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4ec14e100 0x7fb4ec146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48312/0 (socket says 172.21.15.88:48312) 2024-09-21T02:05:35.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.811+0000 7fb4f14f4640 1 -- 172.21.15.88:0/203735699 learned_addr learned my addr 172.21.15.88:0/203735699 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:35.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.811+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb4ec0c4300 con 0x7fb4ec14dcf0 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.812+0000 7fb4f14f4640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4ec14dcf0 msgr2=0x7fb4ec1464e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.812+0000 7fb4f14f4640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4ec14dcf0 0x7fb4ec1464e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.812+0000 7fb4f14f4640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4dc003a00 msgr2=0x7fb4ec151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.812+0000 7fb4f14f4640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4dc003a00 0x7fb4ec151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.812+0000 7fb4f14f4640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec146f60 con 0x7fb4ec14e100 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.813+0000 7fb4f14f4640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4ec14e100 0x7fb4ec146a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0c5180 tx=0x7fb4e400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.813+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4003d80 con 0x7fb4ec14e100 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.813+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4e4005070 con 0x7fb4ec14e100 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.813+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ec151960 con 0x7fb4ec14e100 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.813+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40054e0 con 0x7fb4ec14e100 2024-09-21T02:05:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.813+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ec158030 con 0x7fb4ec14e100 2024-09-21T02:05:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.815+0000 7fb4f2f57640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5460 0x7fb4ec1f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.815+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec151e30 con 0x7fb4ec1f5460 2024-09-21T02:05:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.816+0000 7fb4f0cf3640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5460 0x7fb4ec1f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.817+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb4e4011070 con 0x7fb4ec14e100 2024-09-21T02:05:35.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.819+0000 7fb4f0cf3640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4ec0cb6c0 con 0x7fb4ec1f5460 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.820+0000 7fb4e97fa640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb4d0078b60 0x7fb4d007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.820+0000 7fb4f0cf3640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5460 0x7fb4ec1f5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099a0 tx=0x7fb4e0032cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb4e4005680 con 0x7fb4ec14e100 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0040020 con 0x7fb4ec1f5460 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb4e003c610 con 0x7fb4ec1f5460 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5460 msgr2=0x7fb4ec1f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4ebfff640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb4d0078b60 0x7fb4d007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4e97fa640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5460 0x7fb4ec1f5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099a0 tx=0x7fb4e0032cc0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5460 msgr2=0x7fb4ec1f5870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.822+0000 7fb4e97fa640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5460 0x7fb4ec1f5870 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099a0 tx=0x7fb4e0032cc0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.824+0000 7fb4ebfff640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb4d0078b60 0x7fb4d007b070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8001eb0 tx=0x7fb4d8001600 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.831+0000 7fb4f2f57640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5db0 0x7fb4ec14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:35.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.831+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec157e90 con 0x7fb4ec1f5db0 2024-09-21T02:05:35.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.832+0000 7fb4f0cf3640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5db0 0x7fb4ec14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:35.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.833+0000 7fb4f0cf3640 1 -- 172.21.15.88:0/203735699 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb4ec0c6540 con 0x7fb4ec1f5db0 2024-09-21T02:05:35.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.834+0000 7fb4f0cf3640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5db0 0x7fb4ec14c990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099d0 tx=0x7fb4e0005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:35.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.837+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0040040 con 0x7fb4ec1f5db0 2024-09-21T02:05:35.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5015 (secure 0 0 0) 0x7fb4e003ce10 con 0x7fb4ec1f5db0 2024-09-21T02:05:35.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5db0 msgr2=0x7fb4ec14c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4e97fa640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5db0 0x7fb4ec14c990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00099d0 tx=0x7fb4e0005e30 comp rx=0 tx=0).stop 2024-09-21T02:05:35.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4e97fa640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5db0 msgr2=0x7fb4ec14c990 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:05:35.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4e97fa640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4ec1f5db0 0x7fb4ec14c990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 14, 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 123512, 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: 1 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.840 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:35.841 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:35.842 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:05:35.843 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:05:35.844 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:35.845 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:35.846 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:05:35.847 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:05:35.848 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:05:35.849 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.850 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.851 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:05:35.852 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:05:35.853 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb4d0078b60 msgr2=0x7fb4d007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb4d0078b60 0x7fb4d007b070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8001eb0 tx=0x7fb4d8001600 comp rx=0 tx=0).stop 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4ec14e100 msgr2=0x7fb4ec146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4ec14e100 0x7fb4ec146a20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0c5180 tx=0x7fb4e400c960 comp rx=0 tx=0).stop 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f14f4640 1 -- 172.21.15.88:0/203735699 reap_dead start 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 shutdown_connections 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 --2- 172.21.15.88:0/203735699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4ec14e100 0x7fb4ec146a20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 >> 172.21.15.88:0/203735699 conn(0x7fb4ec071d80 msgr2=0x7fb4ec14f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 shutdown_connections 2024-09-21T02:05:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:35.838+0000 7fb4f2f57640 1 -- 172.21.15.88:0/203735699 wait complete. 2024-09-21T02:05:35.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-21T02:05:35.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-21T02:05:36.109 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:36.109 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 rewriters = 906401.44 kB/sec 2024-09-21T02:05:36.109 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 rewriters = 906251.87 kB/sec 2024-09-21T02:05:36.109 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 906401.44 kB/sec 2024-09-21T02:05:36.109 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 906401.44 kB/sec 2024-09-21T02:05:36.109 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 906401.44 kB/sec 2024-09-21T02:05:36.109 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:38.404 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:38.404 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 readers = 7050244.00 kB/sec 2024-09-21T02:05:38.404 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 readers = 7040492.60 kB/sec 2024-09-21T02:05:38.404 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 7050244.00 kB/sec 2024-09-21T02:05:38.405 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 7050244.00 kB/sec 2024-09-21T02:05:38.405 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 7050244.00 kB/sec 2024-09-21T02:05:38.405 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:40.642 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:40.642 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 re-readers = 6971677.50 kB/sec 2024-09-21T02:05:40.642 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 re-readers = 6961503.00 kB/sec 2024-09-21T02:05:40.642 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 6971677.50 kB/sec 2024-09-21T02:05:40.642 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 6971677.50 kB/sec 2024-09-21T02:05:40.642 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 6971677.50 kB/sec 2024-09-21T02:05:40.643 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:46.965 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:46.965 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:46.966 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:46.966 INFO:tasks.workunit.client.0.smithi088.stdout:iozone test complete. 2024-09-21T02:05:46.966 INFO:tasks.workunit.client.0.smithi088.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-21T02:05:47.031 INFO:tasks.workunit.client.0.smithi088.stdout: Iozone: Performance Test of File I/O 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Version $Revision: 3.494 $ 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Compiled for 64 bit mode. 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Build: linux-AMD64 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T02:05:47.032 INFO:tasks.workunit.client.0.smithi088.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Run began: Sat Sep 21 02:05:46 2024 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Include close in write timing 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Include fsync in write timing 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: File size set to 1048576 kB 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Record Size 1024 kB 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Output is in kBytes/sec 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: Processor cache line size set to 32 bytes. 2024-09-21T02:05:47.033 INFO:tasks.workunit.client.0.smithi088.stdout: File stride size set to 17 * record size. 2024-09-21T02:05:47.034 INFO:tasks.workunit.client.0.smithi088.stdout: Throughput test with 1 process 2024-09-21T02:05:47.034 INFO:tasks.workunit.client.0.smithi088.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-21T02:05:48.468 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:48.468 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 initial writers = 746947.94 kB/sec 2024-09-21T02:05:48.468 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 initial writers = 746827.19 kB/sec 2024-09-21T02:05:48.468 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 746947.94 kB/sec 2024-09-21T02:05:48.468 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 746947.94 kB/sec 2024-09-21T02:05:48.468 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 746947.94 kB/sec 2024-09-21T02:05:48.468 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:52.102 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:52.102 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 rewriters = 698411.38 kB/sec 2024-09-21T02:05:52.102 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 rewriters = 698323.00 kB/sec 2024-09-21T02:05:52.102 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 698411.38 kB/sec 2024-09-21T02:05:52.102 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 698411.38 kB/sec 2024-09-21T02:05:52.102 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 698411.38 kB/sec 2024-09-21T02:05:52.102 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:54.345 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:54.346 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 readers = 7210080.00 kB/sec 2024-09-21T02:05:54.346 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 readers = 7197207.41 kB/sec 2024-09-21T02:05:54.346 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 7210080.00 kB/sec 2024-09-21T02:05:54.346 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 7210080.00 kB/sec 2024-09-21T02:05:54.346 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 7210080.00 kB/sec 2024-09-21T02:05:54.346 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:55.856 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-21T02:05:55.856 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T02:05:55.856 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T02:05:55.859 INFO:tasks.ceph.mon.c:Started 2024-09-21T02:05:55.859 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-21T02:05:55.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:05:55.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 -- 172.21.15.88:0/3598647557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc0c41a0 msgr2=0x7fcadc0c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 --2- 172.21.15.88:0/3598647557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc0c41a0 0x7fcadc0c45b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0c6080 tx=0x7fcad402aac0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 -- 172.21.15.88:0/3598647557 shutdown_connections 2024-09-21T02:05:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 --2- 172.21.15.88:0/3598647557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc0c41a0 0x7fcadc0c45b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 --2- 172.21.15.88:0/3598647557 >> v2:172.21.15.195:3300/0 conn(0x7fcadc153320 0x7fcadc153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 -- 172.21.15.88:0/3598647557 >> 172.21.15.88:0/3598647557 conn(0x7fcadc073210 msgr2=0x7fcadc0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:05:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 -- 172.21.15.88:0/3598647557 shutdown_connections 2024-09-21T02:05:55.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.983+0000 7fcae295e640 1 -- 172.21.15.88:0/3598647557 wait complete. 2024-09-21T02:05:55.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.985+0000 7fcae295e640 1 Processor -- start 2024-09-21T02:05:55.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.985+0000 7fcae295e640 1 -- start start 2024-09-21T02:05:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.986+0000 7fcae295e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcadc0c41a0 0x7fcadc0c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.986+0000 7fcae295e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcadc0c99a0 0x7fcadc1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.986+0000 7fcae295e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc153320 0x7fcadc1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae295e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc070ae0 con 0x7fcadc153320 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae295e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc070dc0 con 0x7fcadc0c99a0 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae295e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcadc0c9db0 con 0x7fcadc0c41a0 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc153320 0x7fcadc1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc153320 0x7fcadc1587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38882/0 (socket says 172.21.15.88:38882) 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 -- 172.21.15.88:0/943334530 learned_addr learned my addr 172.21.15.88:0/943334530 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcadb7fe640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcadc0c99a0 0x7fcadc1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcadbfff640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcadc0c41a0 0x7fcadc0c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 -- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcadc0c41a0 msgr2=0x7fcadc0c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcadc0c41a0 0x7fcadc0c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 -- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcadc0c99a0 msgr2=0x7fcadc1f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcadc0c99a0 0x7fcadc1f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 -- 172.21.15.88:0/943334530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc1f2710 con 0x7fcadc153320 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae0efb640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc153320 0x7fcadc1587e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcad402b0b0 tx=0x7fcad4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcad8ff9640 1 -- 172.21.15.88:0/943334530 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4004250 con 0x7fcadc153320 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcad8ff9640 1 -- 172.21.15.88:0/943334530 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcad40043f0 con 0x7fcadc153320 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcadc0cedb0 con 0x7fcadc153320 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcad8ff9640 1 -- 172.21.15.88:0/943334530 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4004820 con 0x7fcadc153320 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.987+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcadc15b240 con 0x7fcadc153320 2024-09-21T02:05:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.988+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcadc15a1c0 con 0x7fcadc153320 2024-09-21T02:05:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.989+0000 7fcad8ff9640 1 -- 172.21.15.88:0/943334530 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcad4049440 con 0x7fcadc153320 2024-09-21T02:05:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.989+0000 7fcad8ff9640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcac8078b90 0x7fcac807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:05:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.990+0000 7fcad8ff9640 1 -- 172.21.15.88:0/943334530 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcad40bbc70 con 0x7fcadc153320 2024-09-21T02:05:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.990+0000 7fcadbfff640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcac8078b90 0x7fcac807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:05:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.990+0000 7fcadbfff640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcac8078b90 0x7fcac807b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0015e0 tx=0x7fcacc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:05:55.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:55.994+0000 7fcad8ff9640 1 -- 172.21.15.88:0/943334530 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcad4032030 con 0x7fcadc153320 2024-09-21T02:05:56.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:05:56.239+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcadc14e490 con 0x7fcadc153320 2024-09-21T02:05:56.598 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:05:56.598 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 re-readers = 7919530.50 kB/sec 2024-09-21T02:05:56.598 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 re-readers = 7905139.45 kB/sec 2024-09-21T02:05:56.598 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 7919530.50 kB/sec 2024-09-21T02:05:56.599 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 7919530.50 kB/sec 2024-09-21T02:05:56.599 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 7919530.50 kB/sec 2024-09-21T02:05:56.599 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 1048576.00 kB 2024-09-21T02:05:57.711 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:05:57.714 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:06:01.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.175+0000 7fcad8ff9640 1 -- 172.21.15.88:0/943334530 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fcad4084030 con 0x7fcadc153320 2024-09-21T02:06:01.177 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:01.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcac8078b90 msgr2=0x7fcac807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcac8078b90 0x7fcac807b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0015e0 tx=0x7fcacc009040 comp rx=0 tx=0).stop 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc153320 msgr2=0x7fcadc1587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc153320 0x7fcadc1587e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcad402b0b0 tx=0x7fcad4003f10 comp rx=0 tx=0).stop 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 shutdown_connections 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcac8078b90 0x7fcac807b0a0 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0015e0 tx=0x7fcacc009040 comp rx=0 tx=0).stop 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcadc153320 0x7fcadc1587e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcadc0c99a0 0x7fcadc1f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 --2- 172.21.15.88:0/943334530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcadc0c41a0 0x7fcadc0c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.178+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 >> 172.21.15.88:0/943334530 conn(0x7fcadc073210 msgr2=0x7fcadc076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.179+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 shutdown_connections 2024-09-21T02:06:01.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:01.179+0000 7fcae295e640 1 -- 172.21.15.88:0/943334530 wait complete. 2024-09-21T02:06:03.016 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:03.016 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:03.016 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:03.016 INFO:tasks.workunit.client.0.smithi088.stdout:iozone test complete. 2024-09-21T02:06:03.016 INFO:tasks.workunit.client.0.smithi088.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-21T02:06:03.081 INFO:tasks.workunit.client.0.smithi088.stdout: Iozone: Performance Test of File I/O 2024-09-21T02:06:03.081 INFO:tasks.workunit.client.0.smithi088.stdout: Version $Revision: 3.494 $ 2024-09-21T02:06:03.081 INFO:tasks.workunit.client.0.smithi088.stdout: Compiled for 64 bit mode. 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Build: linux-AMD64 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T02:06:03.082 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Run began: Sat Sep 21 02:06:03 2024 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Include close in write timing 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Include fsync in write timing 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: File size set to 10485760 kB 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Record Size 1024 kB 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Output is in kBytes/sec 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Processor cache line size set to 32 bytes. 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: File stride size set to 17 * record size. 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Throughput test with 1 process 2024-09-21T02:06:03.083 INFO:tasks.workunit.client.0.smithi088.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-21T02:06:04.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.315+0000 7f5c68dba640 1 -- 172.21.15.88:0/844695744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64070680 msgr2=0x7f5c64070a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.315+0000 7f5c68dba640 1 --2- 172.21.15.88:0/844695744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64070680 0x7f5c64070a90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5c64073e20 tx=0x7f5c5c02a940 comp rx=0 tx=0).stop 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.316+0000 7f5c68dba640 1 -- 172.21.15.88:0/844695744 shutdown_connections 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.316+0000 7f5c68dba640 1 --2- 172.21.15.88:0/844695744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64070680 0x7f5c64070a90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.316+0000 7f5c68dba640 1 --2- 172.21.15.88:0/844695744 >> v2:172.21.15.195:3300/0 conn(0x7f5c64159c50 0x7f5c6415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.316+0000 7f5c68dba640 1 -- 172.21.15.88:0/844695744 >> 172.21.15.88:0/844695744 conn(0x7f5c640ca8d0 msgr2=0x7f5c640cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.316+0000 7f5c68dba640 1 -- 172.21.15.88:0/844695744 shutdown_connections 2024-09-21T02:06:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.316+0000 7f5c68dba640 1 -- 172.21.15.88:0/844695744 wait complete. 2024-09-21T02:06:04.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.318+0000 7f5c68dba640 1 Processor -- start 2024-09-21T02:06:04.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.318+0000 7f5c68dba640 1 -- start start 2024-09-21T02:06:04.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c68dba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 0x7f5c641558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:04.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c68dba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64142c10 0x7f5c641efff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:04.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c68dba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c64159c50 0x7f5c641565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:04.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c68dba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6415daf0 con 0x7f5c64142c10 2024-09-21T02:06:04.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c68dba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6415d810 con 0x7f5c64159c50 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c68dba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6415a5a0 con 0x7f5c64070680 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c5bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64142c10 0x7f5c641efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c5bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64142c10 0x7f5c641efff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59294/0 (socket says 172.21.15.88:59294) 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c5bfff640 1 -- 172.21.15.88:0/2273589971 learned_addr learned my addr 172.21.15.88:0/2273589971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c62d9d640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c64159c50 0x7f5c641565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c6259c640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 0x7f5c641558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c6259c640 1 -- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 msgr2=0x7f5c641558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c6259c640 1 -- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 msgr2=0x7f5c641558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c6259c640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 0x7f5c641558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c6259c640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 0x7f5c641558c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c5bfff640 1 -- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 msgr2=0x7f5c641558c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c5bfff640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 0x7f5c641558c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c5bfff640 1 -- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c64159c50 msgr2=0x7f5c641565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.319+0000 7f5c5bfff640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c64159c50 0x7f5c641565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.320+0000 7f5c5bfff640 1 -- 172.21.15.88:0/2273589971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c64155e00 con 0x7f5c64142c10 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.320+0000 7f5c5bfff640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64142c10 0x7f5c641efff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00d770 tx=0x7f5c4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.320+0000 7f5c5affd640 1 -- 172.21.15.88:0/2273589971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c017070 con 0x7f5c64142c10 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.320+0000 7f5c5affd640 1 -- 172.21.15.88:0/2273589971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c4c009d20 con 0x7f5c64142c10 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.320+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6415c470 con 0x7f5c64142c10 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.320+0000 7f5c5affd640 1 -- 172.21.15.88:0/2273589971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c004280 con 0x7f5c64142c10 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.320+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6415ca30 con 0x7f5c64142c10 2024-09-21T02:06:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.321+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c640c9280 con 0x7f5c64142c10 2024-09-21T02:06:04.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.323+0000 7f5c5affd640 1 -- 172.21.15.88:0/2273589971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c4c005020 con 0x7f5c64142c10 2024-09-21T02:06:04.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.323+0000 7f5c5affd640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5c44078b90 0x7f5c4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:04.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.323+0000 7f5c5affd640 1 -- 172.21.15.88:0/2273589971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5c4c09b110 con 0x7f5c64142c10 2024-09-21T02:06:04.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.323+0000 7f5c6259c640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5c44078b90 0x7f5c4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:04.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.324+0000 7f5c6259c640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5c44078b90 0x7f5c4407b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c54007cb0 tx=0x7f5c54009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:04.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.328+0000 7f5c5affd640 1 -- 172.21.15.88:0/2273589971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c4c020020 con 0x7f5c64142c10 2024-09-21T02:06:04.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.818+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5c64157170 con 0x7f5c64142c10 2024-09-21T02:06:04.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.820+0000 7f5c5affd640 1 -- 172.21.15.88:0/2273589971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f5c4c063570 con 0x7f5c64142c10 2024-09-21T02:06:04.822 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:04.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.826+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5c44078b90 msgr2=0x7f5c4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:04.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.826+0000 7f5c68dba640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5c44078b90 0x7f5c4407b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c54007cb0 tx=0x7f5c54009040 comp rx=0 tx=0).stop 2024-09-21T02:06:04.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.826+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64142c10 msgr2=0x7f5c641efff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:04.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.826+0000 7f5c68dba640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64142c10 0x7f5c641efff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00d770 tx=0x7f5c4c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 shutdown_connections 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5c44078b90 0x7f5c4407b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c64159c50 0x7f5c641565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c64142c10 0x7f5c641efff0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 --2- 172.21.15.88:0/2273589971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c64070680 0x7f5c641558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 >> 172.21.15.88:0/2273589971 conn(0x7f5c640ca8d0 msgr2=0x7f5c641e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 shutdown_connections 2024-09-21T02:06:04.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:04.827+0000 7f5c68dba640 1 -- 172.21.15.88:0/2273589971 wait complete. 2024-09-21T02:06:04.847 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-21T02:06:04.847 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-21T02:06:04.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:06:05.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.074+0000 7f143d4bd640 1 -- 172.21.15.88:0/4035650334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 msgr2=0x7f143815c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.074+0000 7f143d4bd640 1 --2- 172.21.15.88:0/4035650334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 0x7f143815c150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1430005fd0 tx=0x7f143002a930 comp rx=0 tx=0).stop 2024-09-21T02:06:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.075+0000 7f143d4bd640 1 -- 172.21.15.88:0/4035650334 shutdown_connections 2024-09-21T02:06:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.075+0000 7f143d4bd640 1 --2- 172.21.15.88:0/4035650334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 0x7f143815c150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.075+0000 7f143d4bd640 1 --2- 172.21.15.88:0/4035650334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14380b3ef0 0x7f14380ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.075+0000 7f143d4bd640 1 -- 172.21.15.88:0/4035650334 >> 172.21.15.88:0/4035650334 conn(0x7f1438068d00 msgr2=0x7f1438069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.075+0000 7f143d4bd640 1 -- 172.21.15.88:0/4035650334 shutdown_connections 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.075+0000 7f143d4bd640 1 -- 172.21.15.88:0/4035650334 wait complete. 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.079+0000 7f143d4bd640 1 Processor -- start 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.080+0000 7f143d4bd640 1 -- start start 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.082+0000 7f143d4bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14380b3ef0 0x7f1438151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.082+0000 7f143d4bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 0x7f14381e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f143d4bd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 0x7f14381e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f1436ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14380b3ef0 0x7f1438151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f1436ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14380b3ef0 0x7f1438151920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.88:39350/0 (socket says 172.21.15.88:39350) 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f14377fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 0x7f14381e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f14377fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 0x7f14381e3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59316/0 (socket says 172.21.15.88:59316) 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f1436ffd640 1 -- 172.21.15.88:0/810358998 learned_addr learned my addr 172.21.15.88:0/810358998 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f143815c690 con 0x7f143815bd40 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f14367fc640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 0x7f14381e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14380732f0 con 0x7f14380b3ef0 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f143806fd90 con 0x7f1438155b10 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f14367fc640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 msgr2=0x7f14381e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f14367fc640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 msgr2=0x7f14381e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f14367fc640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 0x7f14381e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.083+0000 7f14367fc640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 0x7f14381e5580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.084+0000 7f1436ffd640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 msgr2=0x7f14381e5580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:05.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.084+0000 7f1436ffd640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1438155b10 0x7f14381e5580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.084+0000 7f1436ffd640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 msgr2=0x7f14381e3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.084+0000 7f1436ffd640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 0x7f14381e3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.084+0000 7f1436ffd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14381e5ac0 con 0x7f14380b3ef0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.084+0000 7f14377fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f143815bd40 0x7f14381e3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.084+0000 7f1436ffd640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14380b3ef0 0x7f1438151920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f142800d950 tx=0x7f142800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.085+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428017070 con 0x7f14380b3ef0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.085+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1428009d40 con 0x7f14380b3ef0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.085+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428005280 con 0x7f14380b3ef0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.085+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14381e7710 con 0x7f14380b3ef0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.085+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14381e66e0 con 0x7f14380b3ef0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.088+0000 7f143d4bd640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14381e5c90 0x7f14381f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.088+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14381e74c0 con 0x7f14381e5c90 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.089+0000 7f14367fc640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14381e5c90 0x7f14381f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.089+0000 7f14367fc640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14380b3d40 con 0x7f14381e5c90 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.090+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1428005420 con 0x7f14380b3ef0 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.091+0000 7f14367fc640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14381e5c90 0x7f14381f9f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1438070550 tx=0x7f142000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.092+0000 7f142f7fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1418078c90 0x7f141807b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.092+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f142809aea0 con 0x7f14380b3ef0 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.093+0000 7f14377fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1418078c90 0x7f141807b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.093+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420005020 con 0x7f14381e5c90 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.093+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f14200051c0 con 0x7f14381e5c90 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.094+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14381e5c90 msgr2=0x7f14381f9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.094+0000 7f142f7fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14381e5c90 0x7f14381f9f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1438070550 tx=0x7f142000e890 comp rx=0 tx=0).stop 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.094+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14381e5c90 msgr2=0x7f14381f9f20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.094+0000 7f142f7fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14381e5c90 0x7f14381f9f20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.098+0000 7f14377fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1418078c90 0x7f141807b1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1430034000 tx=0x7f1430004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.111+0000 7f143d4bd640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1438152500 0x7f14381fa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.111+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14381e6a50 con 0x7f1438152500 2024-09-21T02:06:05.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.111+0000 7f14367fc640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1438152500 0x7f14381fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.112+0000 7f14367fc640 1 -- 172.21.15.88:0/810358998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f143815c8e0 con 0x7f1438152500 2024-09-21T02:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.112+0000 7f14367fc640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1438152500 0x7f14381fa460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1420017000 tx=0x7f1420004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.113+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420013b90 con 0x7f1438152500 2024-09-21T02:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.113+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5052 (secure 0 0 0) 0x7f1420013b90 con 0x7f1438152500 2024-09-21T02:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.113+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1438152500 msgr2=0x7f14381fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.113+0000 7f142f7fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1438152500 0x7f14381fa460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1420017000 tx=0x7f1420004620 comp rx=0 tx=0).stop 2024-09-21T02:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.113+0000 7f142f7fe640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1438152500 msgr2=0x7f14381fa460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:05.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.113+0000 7f142f7fe640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1438152500 0x7f14381fa460 secure :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1420017000 tx=0x7f1420004620 comp rx=0 tx=0).stop 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 20, 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 152829, 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:06:05.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 2, 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:05.120 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:05.121 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:05.122 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:05.123 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:05.124 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:05.125 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.126 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:05.127 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.128 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.129 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:06:05.130 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:05.131 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.132 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.132 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.132 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:05.132 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.116+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1418078c90 msgr2=0x7f141807b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.116+0000 7f143d4bd640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f1418078c90 0x7f141807b1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1430034000 tx=0x7f1430004ab0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.116+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14380b3ef0 msgr2=0x7f1438151920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.116+0000 7f143d4bd640 1 --2- 172.21.15.88:0/810358998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14380b3ef0 0x7f1438151920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f142800d950 tx=0x7f142800de10 comp rx=0 tx=0).stop 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.119+0000 7f14377fe640 1 -- 172.21.15.88:0/810358998 reap_dead start 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.119+0000 7f14377fe640 1 -- 172.21.15.88:0/810358998 reap_dead start 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.120+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 shutdown_connections 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.120+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 >> 172.21.15.88:0/810358998 conn(0x7f1438068d00 msgr2=0x7f143814eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.120+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 shutdown_connections 2024-09-21T02:06:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.120+0000 7f143d4bd640 1 -- 172.21.15.88:0/810358998 wait complete. 2024-09-21T02:06:05.149 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:06:05.149 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-21T02:06:05.149 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:06:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 -- 172.21.15.88:0/1612223799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c2e60 msgr2=0x7f5a4c152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 --2- 172.21.15.88:0/1612223799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c2e60 0x7f5a4c152900 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5a48005fd0 tx=0x7f5a4802a5f0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 -- 172.21.15.88:0/1612223799 shutdown_connections 2024-09-21T02:06:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 --2- 172.21.15.88:0/1612223799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a4c0c7af0 0x7f5a4c0c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 --2- 172.21.15.88:0/1612223799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c2e60 0x7f5a4c152900 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 -- 172.21.15.88:0/1612223799 >> 172.21.15.88:0/1612223799 conn(0x7f5a4c0720b0 msgr2=0x7f5a4c0cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 -- 172.21.15.88:0/1612223799 shutdown_connections 2024-09-21T02:06:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.380+0000 7f5a53e15640 1 -- 172.21.15.88:0/1612223799 wait complete. 2024-09-21T02:06:05.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.384+0000 7f5a53e15640 1 Processor -- start 2024-09-21T02:06:05.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.384+0000 7f5a53e15640 1 -- start start 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.387+0000 7f5a53e15640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a4c0c2e60 0x7f5a4c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.387+0000 7f5a53e15640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c7af0 0x7f5a4c06cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.387+0000 7f5a53e15640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c0c91d0 0x7f5a4c14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.387+0000 7f5a53e15640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c159190 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.387+0000 7f5a513b0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c7af0 0x7f5a4c06cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.387+0000 7f5a513b0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c7af0 0x7f5a4c06cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59344/0 (socket says 172.21.15.88:59344) 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.387+0000 7f5a513b0640 1 -- 172.21.15.88:0/897321578 learned_addr learned my addr 172.21.15.88:0/897321578 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.388+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c159480 con 0x7f5a4c0c91d0 2024-09-21T02:06:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.388+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4c06f1f0 con 0x7f5a4c0c2e60 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a513b0640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a4c0c2e60 msgr2=0x7f5a4c157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a513b0640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a4c0c2e60 0x7f5a4c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a513b0640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c0c91d0 msgr2=0x7f5a4c14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a513b0640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a4c0c91d0 0x7f5a4c14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a513b0640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c0c76e0 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a513b0640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c7af0 0x7f5a4c06cf40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00b440 tx=0x7f5a3c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c015020 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c154370 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c0c7400 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.389+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a3c011920 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.390+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011cd0 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.392+0000 7f5a53e15640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34002a70 0x7f5a34004f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.392+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34007f10 con 0x7f5a34002a70 2024-09-21T02:06:05.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.393+0000 7f5a51bb1640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34002a70 0x7f5a34004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.394+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a3c018020 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.395+0000 7f5a427fc640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5a30078bc0 0x7f5a3007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.395+0000 7f5a51bb1640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a34008110 con 0x7f5a34002a70 2024-09-21T02:06:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.395+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5a3c09ad00 con 0x7f5a4c0c7af0 2024-09-21T02:06:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.396+0000 7f5a51bb1640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34002a70 0x7f5a34004f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0b3e20 tx=0x7f5a480045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.396+0000 7f5a523b2640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5a30078bc0 0x7f5a3007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.397+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4803e030 con 0x7f5a34002a70 2024-09-21T02:06:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.398+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5a480039c0 con 0x7f5a34002a70 2024-09-21T02:06:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.398+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34002a70 msgr2=0x7f5a34004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.398+0000 7f5a427fc640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34002a70 0x7f5a34004f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0b3e20 tx=0x7f5a480045c0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.398+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34002a70 msgr2=0x7f5a34004f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.398+0000 7f5a427fc640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34002a70 0x7f5a34004f80 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0b3e20 tx=0x7f5a480045c0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.399+0000 7f5a523b2640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5a30078bc0 0x7f5a3007b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c073fb0 tx=0x7f5a44009250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.408+0000 7f5a53e15640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34006f30 0x7f5a34007410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.408+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3400a4f0 con 0x7f5a34006f30 2024-09-21T02:06:05.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.409+0000 7f5a51bb1640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34006f30 0x7f5a34007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.410+0000 7f5a51bb1640 1 -- 172.21.15.88:0/897321578 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5a34007950 con 0x7f5a34006f30 2024-09-21T02:06:05.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.411+0000 7f5a51bb1640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34006f30 0x7f5a34007410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009230 tx=0x7f5a48009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.412+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4803e040 con 0x7f5a34006f30 2024-09-21T02:06:05.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.412+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5026 (secure 0 0 0) 0x7f5a4804e450 con 0x7f5a34006f30 2024-09-21T02:06:05.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.412+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34006f30 msgr2=0x7f5a34007410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.412+0000 7f5a427fc640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34006f30 0x7f5a34007410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009230 tx=0x7f5a48009260 comp rx=0 tx=0).stop 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.412+0000 7f5a427fc640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34006f30 msgr2=0x7f5a34007410 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.412+0000 7f5a427fc640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a34006f30 0x7f5a34007410 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 20, 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 153085, 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:05.414 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 3, 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:05.415 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:05.416 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:05.417 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:05.418 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:05.419 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.420 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:05.421 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:05.422 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:05.423 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.424 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.425 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:06:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:05.427 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:05.427 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:05.427 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:05.427 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5a30078bc0 msgr2=0x7f5a3007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5a30078bc0 0x7f5a3007b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c073fb0 tx=0x7f5a44009250 comp rx=0 tx=0).stop 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c7af0 msgr2=0x7f5a4c06cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c7af0 0x7f5a4c06cf40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00b440 tx=0x7f5a3c00b900 comp rx=0 tx=0).stop 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a523b2640 1 -- 172.21.15.88:0/897321578 reap_dead start 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 shutdown_connections 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 --2- 172.21.15.88:0/897321578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a4c0c7af0 0x7f5a4c06cf40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 >> 172.21.15.88:0/897321578 conn(0x7f5a4c0720b0 msgr2=0x7f5a4c14e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 shutdown_connections 2024-09-21T02:06:05.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.413+0000 7f5a53e15640 1 -- 172.21.15.88:0/897321578 wait complete. 2024-09-21T02:06:05.434 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:06:05.434 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-21T02:06:05.434 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-21T02:06:05.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 -- 172.21.15.88:0/465856209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 msgr2=0x7f2e700c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 --2- 172.21.15.88:0/465856209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700c9300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005fd0 tx=0x7f2e6402a670 comp rx=0 tx=0).stop 2024-09-21T02:06:05.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 -- 172.21.15.88:0/465856209 shutdown_connections 2024-09-21T02:06:05.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 --2- 172.21.15.88:0/465856209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700c9300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 --2- 172.21.15.88:0/465856209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e70159460 0x7f2e700c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 -- 172.21.15.88:0/465856209 >> 172.21.15.88:0/465856209 conn(0x7f2e70071570 msgr2=0x7f2e70071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:05.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 -- 172.21.15.88:0/465856209 shutdown_connections 2024-09-21T02:06:05.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.700+0000 7f2e764f9640 1 -- 172.21.15.88:0/465856209 wait complete. 2024-09-21T02:06:05.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.702+0000 7f2e764f9640 1 Processor -- start 2024-09-21T02:06:05.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.702+0000 7f2e764f9640 1 -- start start 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e764f9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e764f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e70152550 0x7f2e700ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e6ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e6ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59356/0 (socket says 172.21.15.88:59356) 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e6ffff640 1 -- 172.21.15.88:0/782478207 learned_addr learned my addr 172.21.15.88:0/782478207 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e764f9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e70153e80 0x7f2e700cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e764f9640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2e700c8390 con 0x7f2e700c6f40 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e764f9640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70158f40 con 0x7f2e70153e80 2024-09-21T02:06:05.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.704+0000 7f2e764f9640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2e700c9be0 con 0x7f2e70152550 2024-09-21T02:06:05.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e70152550 0x7f2e700ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6ffff640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e70152550 msgr2=0x7f2e700ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6ffff640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e70152550 0x7f2e700ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6ffff640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e70153e80 msgr2=0x7f2e700cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6ffff640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e70153e80 0x7f2e700cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6ffff640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e700cbe00 con 0x7f2e700c6f40 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e70152550 0x7f2e700ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.705+0000 7f2e6ffff640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700cb8c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000d770 tx=0x7f2e6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.706+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60017070 con 0x7f2e700c6f40 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.706+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e60009cc0 con 0x7f2e700c6f40 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.706+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600106c0 con 0x7f2e700c6f40 2024-09-21T02:06:05.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.706+0000 7f2e764f9640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7014ab60 con 0x7f2e700c6f40 2024-09-21T02:06:05.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.706+0000 7f2e764f9640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e701496d0 con 0x7f2e700c6f40 2024-09-21T02:06:05.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.708+0000 7f2e764f9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.708+0000 7f2e764f9640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7006a3d0 con 0x7f2e7014a620 2024-09-21T02:06:05.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.709+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2e6001a070 con 0x7f2e700c6f40 2024-09-21T02:06:05.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.709+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.709+0000 7f2e6cff9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2e5c078b90 0x7f2e5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:05.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.710+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2e6009ae40 con 0x7f2e700c6f40 2024-09-21T02:06:05.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.710+0000 7f2e6f7fe640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 msgr2=0x7f2e700cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-21T02:06:05.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.710+0000 7f2e6f7fe640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 msgr2=0x7f2e700cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:06:05.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.710+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:06:05.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.710+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:05.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.712+0000 7f2e74a96640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2e5c078b90 0x7f2e5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.713+0000 7f2e74a96640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2e5c078b90 0x7f2e5c07b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2e680012e0 tx=0x7f2e68007d30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:05.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.912+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:05.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.913+0000 7f2e6f7fe640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 msgr2=0x7f2e700cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-21T02:06:05.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.913+0000 7f2e6f7fe640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 msgr2=0x7f2e700cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:06:05.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.913+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:06:05.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:05.913+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T02:06:06.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.315+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.318+0000 7f2e6f7fe640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e700c9920 con 0x7f2e7014a620 2024-09-21T02:06:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.318+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e700c75b0 tx=0x7f2e640045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:06.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.319+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64035070 con 0x7f2e7014a620 2024-09-21T02:06:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.320+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2e64003940 con 0x7f2e7014a620 2024-09-21T02:06:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.320+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 msgr2=0x7f2e700cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.320+0000 7f2e6cff9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e700c75b0 tx=0x7f2e640045e0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.320+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 msgr2=0x7f2e700cccf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.320+0000 7f2e6cff9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e7014a620 0x7f2e700cccf0 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2e700c75b0 tx=0x7f2e640045e0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.334+0000 7f2e764f9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e701499b0 0x7f2e701f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:06.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.334+0000 7f2e764f9640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7014b9f0 con 0x7f2e701499b0 2024-09-21T02:06:06.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.335+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e701499b0 0x7f2e701f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:06.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.336+0000 7f2e6f7fe640 1 -- 172.21.15.88:0/782478207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2e700c9420 con 0x7f2e701499b0 2024-09-21T02:06:06.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.337+0000 7f2e6f7fe640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e701499b0 0x7f2e701f4c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006010 tx=0x7f2e64009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:06.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.338+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64004e30 con 0x7f2e701499b0 2024-09-21T02:06:06.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.339+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4548 (secure 0 0 0) 0x7f2e6404d070 con 0x7f2e701499b0 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.339+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e701499b0 msgr2=0x7f2e701f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.339+0000 7f2e6cff9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e701499b0 0x7f2e701f4c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006010 tx=0x7f2e64009260 comp rx=0 tx=0).stop 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.339+0000 7f2e6cff9640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e701499b0 msgr2=0x7f2e701f4c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.339+0000 7f2e6cff9640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e701499b0 0x7f2e701f4c90 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006010 tx=0x7f2e64009260 comp rx=0 tx=0).stop 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 20, 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 10380, 2024-09-21T02:06:06.341 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 4, 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:06.342 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:06.343 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:06.344 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:06.345 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:06.346 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:06.347 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.348 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:06.349 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.350 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:06.351 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:06.352 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:06.353 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:06.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.340+0000 7f2e567fc640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2e5c078b90 msgr2=0x7f2e5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:06.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.341+0000 7f2e567fc640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2e5c078b90 0x7f2e5c07b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2e680012e0 tx=0x7f2e68007d30 comp rx=0 tx=0).stop 2024-09-21T02:06:06.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.341+0000 7f2e74a96640 1 -- 172.21.15.88:0/782478207 reap_dead start 2024-09-21T02:06:06.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.341+0000 7f2e567fc640 1 -- 172.21.15.88:0/782478207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 msgr2=0x7f2e700cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:06.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.341+0000 7f2e567fc640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700cb8c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000d770 tx=0x7f2e6000dc30 comp rx=0 tx=0).stop 2024-09-21T02:06:06.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.341+0000 7f2e567fc640 1 -- 172.21.15.88:0/782478207 shutdown_connections 2024-09-21T02:06:06.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.341+0000 7f2e567fc640 1 --2- 172.21.15.88:0/782478207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e700c6f40 0x7f2e700cb8c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.341+0000 7f2e567fc640 1 -- 172.21.15.88:0/782478207 >> 172.21.15.88:0/782478207 conn(0x7f2e70071570 msgr2=0x7f2e70157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:06.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.342+0000 7f2e567fc640 1 -- 172.21.15.88:0/782478207 shutdown_connections 2024-09-21T02:06:06.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.342+0000 7f2e567fc640 1 -- 172.21.15.88:0/782478207 wait complete. 2024-09-21T02:06:06.360 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:06:06.361 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-21T02:06:06.361 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.556+0000 7fc174b4a640 1 -- 172.21.15.88:0/2212036958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cb200 msgr2=0x7fc1700cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.556+0000 7fc174b4a640 1 --2- 172.21.15.88:0/2212036958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cb200 0x7fc1700cb770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc164005fd0 tx=0x7fc16402a5d0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.556+0000 7fc174b4a640 1 -- 172.21.15.88:0/2212036958 shutdown_connections 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.556+0000 7fc174b4a640 1 --2- 172.21.15.88:0/2212036958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cb200 0x7fc1700cb770 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.556+0000 7fc174b4a640 1 --2- 172.21.15.88:0/2212036958 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc17015f9c0 0x7fc170155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.556+0000 7fc174b4a640 1 -- 172.21.15.88:0/2212036958 >> 172.21.15.88:0/2212036958 conn(0x7fc170073210 msgr2=0x7fc1700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.557+0000 7fc174b4a640 1 -- 172.21.15.88:0/2212036958 shutdown_connections 2024-09-21T02:06:06.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.557+0000 7fc174b4a640 1 -- 172.21.15.88:0/2212036958 wait complete. 2024-09-21T02:06:06.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.559+0000 7fc174b4a640 1 Processor -- start 2024-09-21T02:06:06.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.559+0000 7fc174b4a640 1 -- start start 2024-09-21T02:06:06.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.560+0000 7fc174b4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1700cb200 0x7fc1701f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:06.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.560+0000 7fc174b4a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cbcb0 0x7fc17014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:06.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc174b4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17015f9c0 0x7fc170157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc174b4a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc1700c9e20 con 0x7fc1700cbcb0 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc174b4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc1700cc270 con 0x7fc17015f9c0 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc174b4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc1700cabf0 con 0x7fc1700cb200 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cbcb0 0x7fc17014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cbcb0 0x7fc17014bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59364/0 (socket says 172.21.15.88:59364) 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 -- 172.21.15.88:0/1816128091 learned_addr learned my addr 172.21.15.88:0/1816128091 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16e59c640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1700cb200 0x7fc1701f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 -- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1700cb200 msgr2=0x7fc1701f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16ed9d640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17015f9c0 0x7fc170157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1700cb200 0x7fc1701f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 -- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17015f9c0 msgr2=0x7fc170157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17015f9c0 0x7fc170157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16dd9b640 1 -- 172.21.15.88:0/1816128091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc170158ff0 con 0x7fc1700cbcb0 2024-09-21T02:06:06.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16e59c640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1700cb200 0x7fc1701f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.561+0000 7fc16ed9d640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17015f9c0 0x7fc170157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.562+0000 7fc16dd9b640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cbcb0 0x7fc17014bb20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc164006010 tx=0x7fc1640038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.562+0000 7fc15effd640 1 -- 172.21.15.88:0/1816128091 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16403e030 con 0x7fc1700cbcb0 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.562+0000 7fc15effd640 1 -- 172.21.15.88:0/1816128091 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc164003ca0 con 0x7fc1700cbcb0 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.562+0000 7fc15effd640 1 -- 172.21.15.88:0/1816128091 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164035040 con 0x7fc1700cbcb0 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.562+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1701f6180 con 0x7fc1700cbcb0 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.562+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1700cd410 con 0x7fc1700cbcb0 2024-09-21T02:06:06.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.564+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17006ef40 con 0x7fc1700cbcb0 2024-09-21T02:06:06.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.565+0000 7fc15effd640 1 -- 172.21.15.88:0/1816128091 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc164041030 con 0x7fc1700cbcb0 2024-09-21T02:06:06.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.565+0000 7fc15effd640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc150078b90 0x7fc15007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:06.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.565+0000 7fc15effd640 1 -- 172.21.15.88:0/1816128091 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc1640bb2d0 con 0x7fc1700cbcb0 2024-09-21T02:06:06.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.566+0000 7fc16e59c640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc150078b90 0x7fc15007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:06.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.567+0000 7fc16e59c640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc150078b90 0x7fc15007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc16000a870 tx=0x7fc160008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:06.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:06.574+0000 7fc15effd640 1 -- 172.21.15.88:0/1816128091 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc17006ef40 con 0x7fc1700cbcb0 2024-09-21T02:06:07.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.097+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fc170071a50 con 0x7fc1700cbcb0 2024-09-21T02:06:07.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.098+0000 7fc15effd640 1 -- 172.21.15.88:0/1816128091 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fc170071a50 con 0x7fc1700cbcb0 2024-09-21T02:06:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc150078b90 msgr2=0x7fc15007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc150078b90 0x7fc15007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc16000a870 tx=0x7fc160008040 comp rx=0 tx=0).stop 2024-09-21T02:06:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cbcb0 msgr2=0x7fc17014bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cbcb0 0x7fc17014bb20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc164006010 tx=0x7fc1640038a0 comp rx=0 tx=0).stop 2024-09-21T02:06:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 shutdown_connections 2024-09-21T02:06:07.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc150078b90 0x7fc15007b0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc17015f9c0 0x7fc170157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1700cbcb0 0x7fc17014bb20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 --2- 172.21.15.88:0/1816128091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1700cb200 0x7fc1701f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 >> 172.21.15.88:0/1816128091 conn(0x7fc170073210 msgr2=0x7fc170159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 shutdown_connections 2024-09-21T02:06:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:07.103+0000 7fc174b4a640 1 -- 172.21.15.88:0/1816128091 wait complete. 2024-09-21T02:06:07.119 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-21T02:06:14.930 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:14.930 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 initial writers = 887373.62 kB/sec 2024-09-21T02:06:14.931 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 initial writers = 887353.83 kB/sec 2024-09-21T02:06:14.931 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 887373.62 kB/sec 2024-09-21T02:06:14.931 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 887373.62 kB/sec 2024-09-21T02:06:14.931 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 887373.62 kB/sec 2024-09-21T02:06:14.931 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 10485760.00 kB 2024-09-21T02:06:17.120 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-21T02:06:17.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:06:17.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.255+0000 7f62c34cc640 1 -- 172.21.15.88:0/220182543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 msgr2=0x7f62a8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.255+0000 7f62c34cc640 1 --2- 172.21.15.88:0/220182543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 0x7f62a8003ea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f62b80016c0 tx=0x7f62a8005830 comp rx=0 tx=0).stop 2024-09-21T02:06:17.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.255+0000 7f62c1a69640 1 -- 172.21.15.88:0/220182543 reap_dead start 2024-09-21T02:06:17.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.255+0000 7f62c34cc640 1 -- 172.21.15.88:0/220182543 shutdown_connections 2024-09-21T02:06:17.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.255+0000 7f62c34cc640 1 -- 172.21.15.88:0/220182543 >> 172.21.15.88:0/220182543 conn(0x7f62bc0ca9b0 msgr2=0x7f62bc0cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:17.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.255+0000 7f62c34cc640 1 -- 172.21.15.88:0/220182543 shutdown_connections 2024-09-21T02:06:17.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.255+0000 7f62c34cc640 1 -- 172.21.15.88:0/220182543 wait complete. 2024-09-21T02:06:17.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.258+0000 7f62c34cc640 1 Processor -- start 2024-09-21T02:06:17.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.258+0000 7f62c34cc640 1 -- start start 2024-09-21T02:06:17.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c34cc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 0x7f62bc151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c34cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc14dab0 0x7f62bc14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c34cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc14dec0 0x7f62bc14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c34cc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc074490 con 0x7f62a8003a30 2024-09-21T02:06:17.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c34cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc15bee0 con 0x7f62bc14dab0 2024-09-21T02:06:17.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c34cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f62bc0ca650 con 0x7f62bc14dec0 2024-09-21T02:06:17.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 0x7f62bc151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 0x7f62bc151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45266/0 (socket says 172.21.15.88:45266) 2024-09-21T02:06:17.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 -- 172.21.15.88:0/3413707520 learned_addr learned my addr 172.21.15.88:0/3413707520 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:17.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1a69640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc14dec0 0x7f62bc14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c0a67640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc14dab0 0x7f62bc14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 -- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc14dec0 msgr2=0x7f62bc14cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc14dec0 0x7f62bc14cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 -- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc14dab0 msgr2=0x7f62bc14ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc14dab0 0x7f62bc14ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1268640 1 -- 172.21.15.88:0/3413707520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc151aa0 con 0x7f62a8003a30 2024-09-21T02:06:17.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c1a69640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc14dec0 0x7f62bc14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:17.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.261+0000 7f62c0a67640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc14dab0 0x7f62bc14ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:06:17.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.262+0000 7f62c1268640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 0x7f62bc151560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f62b802a930 tx=0x7f62b8006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:17.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.262+0000 7f62b1ffb640 1 -- 172.21.15.88:0/3413707520 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8041070 con 0x7f62a8003a30 2024-09-21T02:06:17.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.262+0000 7f62b1ffb640 1 -- 172.21.15.88:0/3413707520 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62b8003af0 con 0x7f62a8003a30 2024-09-21T02:06:17.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.262+0000 7f62b1ffb640 1 -- 172.21.15.88:0/3413707520 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8038070 con 0x7f62a8003a30 2024-09-21T02:06:17.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.262+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc152010 con 0x7f62a8003a30 2024-09-21T02:06:17.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.262+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc14e7b0 con 0x7f62a8003a30 2024-09-21T02:06:17.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.263+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62bc074050 con 0x7f62a8003a30 2024-09-21T02:06:17.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.264+0000 7f62b1ffb640 1 -- 172.21.15.88:0/3413707520 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62b8044070 con 0x7f62a8003a30 2024-09-21T02:06:17.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.264+0000 7f62b1ffb640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a0078b90 0x7f62a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.264+0000 7f62b1ffb640 1 -- 172.21.15.88:0/3413707520 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f62b80ba4b0 con 0x7f62a8003a30 2024-09-21T02:06:17.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.265+0000 7f62c0a67640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a0078b90 0x7f62a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.265+0000 7f62c0a67640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a0078b90 0x7f62a007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62ac001eb0 tx=0x7f62ac001110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:17.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.269+0000 7f62b1ffb640 1 -- 172.21.15.88:0/3413707520 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62b8006500 con 0x7f62a8003a30 2024-09-21T02:06:17.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.528+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f62bc074c00 con 0x7f62a8003a30 2024-09-21T02:06:17.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.529+0000 7f62b1ffb640 1 -- 172.21.15.88:0/3413707520 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f62b8082990 con 0x7f62a8003a30 2024-09-21T02:06:17.530 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a0078b90 msgr2=0x7f62a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a0078b90 0x7f62a007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62ac001eb0 tx=0x7f62ac001110 comp rx=0 tx=0).stop 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 msgr2=0x7f62bc151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 0x7f62bc151560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f62b802a930 tx=0x7f62b8006310 comp rx=0 tx=0).stop 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 shutdown_connections 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f62a0078b90 0x7f62a007b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62bc14dec0 0x7f62bc14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62bc14dab0 0x7f62bc14ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 --2- 172.21.15.88:0/3413707520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62a8003a30 0x7f62bc151560 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.531+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 >> 172.21.15.88:0/3413707520 conn(0x7f62bc0ca9b0 msgr2=0x7f62bc14eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:17.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.532+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 shutdown_connections 2024-09-21T02:06:17.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.532+0000 7f62c34cc640 1 -- 172.21.15.88:0/3413707520 wait complete. 2024-09-21T02:06:17.546 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-21T02:06:17.546 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-21T02:06:17.546 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-21T02:06:17.546 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:06:17.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 -- 172.21.15.88:0/3455654647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c97d0 msgr2=0x7f6398156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 --2- 172.21.15.88:0/3455654647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c97d0 0x7f6398156d60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f63980ced50 tx=0x7f638002a560 comp rx=0 tx=0).stop 2024-09-21T02:06:17.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 -- 172.21.15.88:0/3455654647 shutdown_connections 2024-09-21T02:06:17.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 --2- 172.21.15.88:0/3455654647 >> v2:172.21.15.195:3300/0 conn(0x7f63980c76c0 0x7f63981640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 --2- 172.21.15.88:0/3455654647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c97d0 0x7f6398156d60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 -- 172.21.15.88:0/3455654647 >> 172.21.15.88:0/3455654647 conn(0x7f63980720b0 msgr2=0x7f63980cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:17.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 -- 172.21.15.88:0/3455654647 shutdown_connections 2024-09-21T02:06:17.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.673+0000 7f639e5c3640 1 -- 172.21.15.88:0/3455654647 wait complete. 2024-09-21T02:06:17.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.675+0000 7f639e5c3640 1 Processor -- start 2024-09-21T02:06:17.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.675+0000 7f639e5c3640 1 -- start start 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.676+0000 7f639e5c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63980c72b0 0x7f6398152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.676+0000 7f639e5c3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c76c0 0x7f639806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.676+0000 7f639e5c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63980c97d0 0x7f63981e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.676+0000 7f639e5c3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63980c7ad0 con 0x7f63980c76c0 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.676+0000 7f639e5c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6398159850 con 0x7f63980c97d0 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.676+0000 7f639e5c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f63980c9d00 con 0x7f63980c72b0 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c76c0 0x7f639806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c76c0 0x7f639806e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45294/0 (socket says 172.21.15.88:45294) 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 -- 172.21.15.88:0/1924910369 learned_addr learned my addr 172.21.15.88:0/1924910369 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f639cb60640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63980c97d0 0x7f63981e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f6397fff640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63980c72b0 0x7f6398152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63980c72b0 msgr2=0x7f6398152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63980c72b0 0x7f6398152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63980c97d0 msgr2=0x7f63981e4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63980c97d0 0x7f63981e4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 -- 172.21.15.88:0/1924910369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639806ec10 con 0x7f63980c76c0 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f63977fe640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c76c0 0x7f639806e6d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f638800c3d0 tx=0x7f638800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388017070 con 0x7f63980c76c0 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6398151730 con 0x7f63980c76c0 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63880053c0 con 0x7f63980c76c0 2024-09-21T02:06:17.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.677+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388003850 con 0x7f63980c76c0 2024-09-21T02:06:17.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.678+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6398151cf0 con 0x7f63980c76c0 2024-09-21T02:06:17.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.679+0000 7f639e5c3640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639806df60 0x7f639814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.679+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63981eff20 con 0x7f639806df60 2024-09-21T02:06:17.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.679+0000 7f6397fff640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639806df60 0x7f639814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.679+0000 7f6397fff640 1 -- 172.21.15.88:0/1924910369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63981514a0 con 0x7f639806df60 2024-09-21T02:06:17.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.679+0000 7f6397fff640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639806df60 0x7f639814c850 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6398073ce0 tx=0x7f63800035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.679+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380040070 con 0x7f639806df60 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6380003cd0 con 0x7f639806df60 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639806df60 msgr2=0x7f639814c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639806df60 0x7f639814c850 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6398073ce0 tx=0x7f63800035f0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639806df60 msgr2=0x7f639814c850 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639806df60 0x7f639814c850 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f638801a070 con 0x7f63980c76c0 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6378078540 0x7f637807aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f638809a0a0 con 0x7f63980c76c0 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.680+0000 7f6397fff640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6378078540 0x7f637807aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.681+0000 7f6397fff640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6378078540 0x7f637807aa50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63981e6cd0 tx=0x7f638003f000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:17.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.694+0000 7f639e5c3640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639814cd90 0x7f63981e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:17.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.694+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63980758e0 con 0x7f639814cd90 2024-09-21T02:06:17.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.694+0000 7f639cb60640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639814cd90 0x7f63981e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:17.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.694+0000 7f639cb60640 1 -- 172.21.15.88:0/1924910369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6398153590 con 0x7f639814cd90 2024-09-21T02:06:17.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.694+0000 7f639cb60640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639814cd90 0x7f63981e2b60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6390005fd0 tx=0x7f6390010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:17.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390012450 con 0x7f639814cd90 2024-09-21T02:06:17.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5053 (secure 0 0 0) 0x7f6390012450 con 0x7f639814cd90 2024-09-21T02:06:17.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639814cd90 msgr2=0x7f63981e2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f6394ff9640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639814cd90 0x7f63981e2b60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6390005fd0 tx=0x7f6390010f50 comp rx=0 tx=0).stop 2024-09-21T02:06:17.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f6394ff9640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639814cd90 msgr2=0x7f63981e2b60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f6394ff9640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f639814cd90 0x7f63981e2b60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 20, 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 165411, 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:06:17.697 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 15, 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:17.698 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:17.699 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:17.700 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:17.701 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:17.702 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:17.703 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:17.704 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:17.705 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:17.706 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:17.707 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:17.708 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:17.709 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6378078540 msgr2=0x7f637807aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639e5c3640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6378078540 0x7f637807aa50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63981e6cd0 tx=0x7f638003f000 comp rx=0 tx=0).stop 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c76c0 msgr2=0x7f639806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639e5c3640 1 --2- 172.21.15.88:0/1924910369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63980c76c0 0x7f639806e6d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f638800c3d0 tx=0x7f638800c890 comp rx=0 tx=0).stop 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639cb60640 1 -- 172.21.15.88:0/1924910369 reap_dead start 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639cb60640 1 -- 172.21.15.88:0/1924910369 reap_dead start 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 shutdown_connections 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 >> 172.21.15.88:0/1924910369 conn(0x7f63980720b0 msgr2=0x7f63981584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.695+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 shutdown_connections 2024-09-21T02:06:17.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.696+0000 7f639e5c3640 1 -- 172.21.15.88:0/1924910369 wait complete. 2024-09-21T02:06:17.712 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:06:17.712 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-21T02:06:17.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:06:18.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.979+0000 7f8a75b2b640 1 -- 172.21.15.88:0/1162123818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 msgr2=0x7f8a58004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.979+0000 7f8a75b2b640 1 --2- 172.21.15.88:0/1162123818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 0x7f8a58004c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c710 tx=0x7f8a58005830 comp rx=0 tx=0).stop 2024-09-21T02:06:18.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.979+0000 7f8a6ffff640 1 -- 172.21.15.88:0/1162123818 reap_dead start 2024-09-21T02:06:18.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.990+0000 7f8a75b2b640 1 -- 172.21.15.88:0/1162123818 shutdown_connections 2024-09-21T02:06:18.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:17.990+0000 7f8a75b2b640 1 -- 172.21.15.88:0/1162123818 >> 172.21.15.88:0/1162123818 conn(0x7f8a700720b0 msgr2=0x7f8a700cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:18.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.003+0000 7f8a75b2b640 1 -- 172.21.15.88:0/1162123818 shutdown_connections 2024-09-21T02:06:18.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.003+0000 7f8a75b2b640 1 -- 172.21.15.88:0/1162123818 wait complete. 2024-09-21T02:06:18.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.007+0000 7f8a75b2b640 1 Processor -- start 2024-09-21T02:06:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.011+0000 7f8a75b2b640 1 -- start start 2024-09-21T02:06:18.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.016+0000 7f8a75b2b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 0x7f8a70152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.016+0000 7f8a6f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 0x7f8a70152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.017+0000 7f8a6f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 0x7f8a70152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45340/0 (socket says 172.21.15.88:45340) 2024-09-21T02:06:18.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.017+0000 7f8a6f7fe640 1 -- 172.21.15.88:0/852079040 learned_addr learned my addr 172.21.15.88:0/852079040 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:18.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.017+0000 7f8a75b2b640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7015cef0 0x7f8a7015d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.018+0000 7f8a75b2b640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7015ee30 0x7f8a7015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.019+0000 7f8a6effd640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7015cef0 0x7f8a7015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.022+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a700b40d0 con 0x7f8a58004800 2024-09-21T02:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.022+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7015bbb0 con 0x7f8a7015cef0 2024-09-21T02:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.022+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f8a700b4250 con 0x7f8a7015ee30 2024-09-21T02:06:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.024+0000 7f8a6ffff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7015ee30 0x7f8a7015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.026+0000 7f8a6effd640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7015ee30 msgr2=0x7f8a7015f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.026+0000 7f8a6effd640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7015ee30 0x7f8a7015f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.026+0000 7f8a6effd640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 msgr2=0x7f8a70152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.026+0000 7f8a6effd640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 0x7f8a70152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.026+0000 7f8a6effd640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a701537d0 con 0x7f8a7015cef0 2024-09-21T02:06:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.026+0000 7f8a6f7fe640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a58004800 0x7f8a70152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:18.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.027+0000 7f8a6effd640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7015cef0 0x7f8a7015d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a6402f000 tx=0x7f8a640065b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.028+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6402add0 con 0x7f8a7015cef0 2024-09-21T02:06:18.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.028+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a640046a0 con 0x7f8a7015cef0 2024-09-21T02:06:18.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.028+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64004af0 con 0x7f8a7015cef0 2024-09-21T02:06:18.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.028+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a70153530 con 0x7f8a7015cef0 2024-09-21T02:06:18.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.028+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a701fa1d0 con 0x7f8a7015cef0 2024-09-21T02:06:18.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.032+0000 7f8a4ffff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6900 0x7f8a70159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.033+0000 7f8a4ffff640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70154570 con 0x7f8a701f6900 2024-09-21T02:06:18.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.036+0000 7f8a6f7fe640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6900 0x7f8a70159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.039+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a64043020 con 0x7f8a7015cef0 2024-09-21T02:06:18.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.041+0000 7f8a57fff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8a5c078be0 0x7f8a5c07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.041+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8a640ba9c0 con 0x7f8a7015cef0 2024-09-21T02:06:18.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.041+0000 7f8a6f7fe640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a700c48d0 con 0x7f8a701f6900 2024-09-21T02:06:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.043+0000 7f8a6ffff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8a5c078be0 0x7f8a5c07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.048+0000 7f8a6ffff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8a5c078be0 0x7f8a5c07b0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a70075910 tx=0x7f8a68001de0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.049+0000 7f8a6f7fe640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6900 0x7f8a70159720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000eb10 tx=0x7f8a60009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.051+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60017070 con 0x7f8a701f6900 2024-09-21T02:06:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.051+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8a6000ca50 con 0x7f8a701f6900 2024-09-21T02:06:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.051+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6900 msgr2=0x7f8a70159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.051+0000 7f8a57fff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6900 0x7f8a70159720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000eb10 tx=0x7f8a60009e90 comp rx=0 tx=0).stop 2024-09-21T02:06:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.051+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6900 msgr2=0x7f8a70159720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:18.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.051+0000 7f8a57fff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6900 0x7f8a70159720 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000eb10 tx=0x7f8a60009e90 comp rx=0 tx=0).stop 2024-09-21T02:06:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.114+0000 7f8a4ffff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6d10 0x7f8a70159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.114+0000 7f8a4ffff640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a700c80d0 con 0x7f8a701f6d10 2024-09-21T02:06:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.116+0000 7f8a6f7fe640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6d10 0x7f8a70159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.118+0000 7f8a6f7fe640 1 -- 172.21.15.88:0/852079040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8a70074c70 con 0x7f8a701f6d10 2024-09-21T02:06:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.120+0000 7f8a6f7fe640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6d10 0x7f8a70159c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a60019000 tx=0x7f8a60004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.122+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60017070 con 0x7f8a701f6d10 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.123+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5027 (secure 0 0 0) 0x7f8a600064d0 con 0x7f8a701f6d10 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.123+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6d10 msgr2=0x7f8a70159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.123+0000 7f8a57fff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6d10 0x7f8a70159c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a60019000 tx=0x7f8a60004770 comp rx=0 tx=0).stop 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.123+0000 7f8a57fff640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6d10 msgr2=0x7f8a70159c60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.123+0000 7f8a57fff640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a701f6d10 0x7f8a70159c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:18.124 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 20, 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 165795, 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 15, 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.129 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.130 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:18.131 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:18.132 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:18.133 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.134 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:18.135 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.136 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.137 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:18.137 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:18.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8a5c078be0 msgr2=0x7f8a5c07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a75b2b640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f8a5c078be0 0x7f8a5c07b0f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8a70075910 tx=0x7f8a68001de0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a6ffff640 1 -- 172.21.15.88:0/852079040 reap_dead start 2024-09-21T02:06:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7015cef0 msgr2=0x7f8a7015d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a75b2b640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7015cef0 0x7f8a7015d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a6402f000 tx=0x7f8a640065b0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 shutdown_connections 2024-09-21T02:06:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a75b2b640 1 --2- 172.21.15.88:0/852079040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7015cef0 0x7f8a7015d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.124+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 >> 172.21.15.88:0/852079040 conn(0x7f8a700720b0 msgr2=0x7f8a70073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.125+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 shutdown_connections 2024-09-21T02:06:18.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.125+0000 7f8a75b2b640 1 -- 172.21.15.88:0/852079040 wait complete. 2024-09-21T02:06:18.156 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:06:18.156 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-21T02:06:18.156 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-21T02:06:18.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.414+0000 7f1015239640 1 -- 172.21.15.88:0/497148678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 msgr2=0x7f0ff8005d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.414+0000 7f1015239640 1 --2- 172.21.15.88:0/497148678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f0ff8005d20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f100000bbb0 tx=0x7f0ff8006940 comp rx=0 tx=0).stop 2024-09-21T02:06:18.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.415+0000 7f1015239640 1 -- 172.21.15.88:0/497148678 shutdown_connections 2024-09-21T02:06:18.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.415+0000 7f1015239640 1 --2- 172.21.15.88:0/497148678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f0ff8005d20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.415+0000 7f1015239640 1 -- 172.21.15.88:0/497148678 >> 172.21.15.88:0/497148678 conn(0x7f101006cf70 msgr2=0x7f101006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:18.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.415+0000 7f1015239640 1 -- 172.21.15.88:0/497148678 shutdown_connections 2024-09-21T02:06:18.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.416+0000 7f1015239640 1 -- 172.21.15.88:0/497148678 wait complete. 2024-09-21T02:06:18.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.419+0000 7f1015239640 1 Processor -- start 2024-09-21T02:06:18.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.419+0000 7f1015239640 1 -- start start 2024-09-21T02:06:18.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f1015239640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f1010156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f1015239640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10101423f0 0x7f1010142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f100ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f1010156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f100ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f1010156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38914/0 (socket says 172.21.15.88:38914) 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f1015239640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010149a70 0x7f1010149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f100ffff640 1 -- 172.21.15.88:0/3618391828 learned_addr learned my addr 172.21.15.88:0/3618391828 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1010074610 con 0x7f0ff8005910 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f10100b29f0 con 0x7f10101423f0 2024-09-21T02:06:18.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.422+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1010074790 con 0x7f1010149a70 2024-09-21T02:06:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.423+0000 7f100ffff640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010149a70 msgr2=0x7f1010149e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:06:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.423+0000 7f100ffff640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010149a70 0x7f1010149e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.423+0000 7f100ffff640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10101423f0 msgr2=0x7f1010142800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:06:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.423+0000 7f100ffff640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f10101423f0 0x7f1010142800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.423+0000 7f100ffff640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101014a3c0 con 0x7f0ff8005910 2024-09-21T02:06:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.424+0000 7f100ffff640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f1010156b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f100400c720 tx=0x7f100400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.424+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004017070 con 0x7f0ff8005910 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.424+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f100400ce30 con 0x7f0ff8005910 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.424+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100400a880 con 0x7f0ff8005910 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.424+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10101570c0 con 0x7f0ff8005910 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.424+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10101edf30 con 0x7f0ff8005910 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.426+0000 7f1015239640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010155dc0 0x7f10101561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.426+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1010158150 con 0x7f1010155dc0 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.426+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f100401a070 con 0x7f0ff8005910 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.427+0000 7f100cff9640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0ffc078c10 0x7f0ffc07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.427+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f100409a2a0 con 0x7f0ff8005910 2024-09-21T02:06:18.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.428+0000 7f100f7fe640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010155dc0 0x7f10101561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.429+0000 7f1014a38640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0ffc078c10 0x7f0ffc07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.431+0000 7f100f7fe640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1010075660 con 0x7f1010155dc0 2024-09-21T02:06:18.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.432+0000 7f100f7fe640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010155dc0 0x7f10101561d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10100c2a90 tx=0x7f1000003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.432+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000035030 con 0x7f1010155dc0 2024-09-21T02:06:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.433+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1000003cf0 con 0x7f1010155dc0 2024-09-21T02:06:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.433+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010155dc0 msgr2=0x7f10101561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.433+0000 7f100cff9640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010155dc0 0x7f10101561d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10100c2a90 tx=0x7f1000003820 comp rx=0 tx=0).stop 2024-09-21T02:06:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.433+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010155dc0 msgr2=0x7f10101561d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:18.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.433+0000 7f100cff9640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1010155dc0 0x7f10101561d0 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f10100c2a90 tx=0x7f1000003820 comp rx=0 tx=0).stop 2024-09-21T02:06:18.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.434+0000 7f1014a38640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0ffc078c10 0x7f0ffc07b120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f10100c2820 tx=0x7f1008007d30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.440+0000 7f1015239640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10101f1200 0x7f10101f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.440+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f1010155b00 con 0x7f10101f1200 2024-09-21T02:06:18.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.441+0000 7f100f7fe640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10101f1200 0x7f10101f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.444+0000 7f100f7fe640 1 -- 172.21.15.88:0/3618391828 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f10100b25c0 con 0x7f10101f1200 2024-09-21T02:06:18.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.446+0000 7f100f7fe640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10101f1200 0x7f10101f1610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f100000bb00 tx=0x7f0ff8006940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.446+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000035040 con 0x7f10101f1200 2024-09-21T02:06:18.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.447+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4549 (secure 0 0 0) 0x7f1000045c00 con 0x7f10101f1200 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.447+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10101f1200 msgr2=0x7f10101f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.447+0000 7f100cff9640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10101f1200 0x7f10101f1610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f100000bb00 tx=0x7f0ff8006940 comp rx=0 tx=0).stop 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.447+0000 7f100cff9640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10101f1200 msgr2=0x7f10101f1610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.447+0000 7f100cff9640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10101f1200 0x7f10101f1610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 20, 2024-09-21T02:06:18.449 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 22488, 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 16, 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:18.450 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:18.451 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:18.452 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:18.453 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:18.454 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.455 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:18.456 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.457 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:18.458 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:18.459 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:18.460 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0ffc078c10 msgr2=0x7f0ffc07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0ffc078c10 0x7f0ffc07b120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f10100c2820 tx=0x7f1008007d30 comp rx=0 tx=0).stop 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 msgr2=0x7f1010156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f1010156b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f100400c720 tx=0x7f100400cbe0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1014a38640 1 -- 172.21.15.88:0/3618391828 reap_dead start 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 shutdown_connections 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 --2- 172.21.15.88:0/3618391828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ff8005910 0x7f1010156b80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 >> 172.21.15.88:0/3618391828 conn(0x7f101006cf70 msgr2=0x7f1010144620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:18.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 shutdown_connections 2024-09-21T02:06:18.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.448+0000 7f1015239640 1 -- 172.21.15.88:0/3618391828 wait complete. 2024-09-21T02:06:18.465 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:06:18.465 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-21T02:06:18.465 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-21T02:06:18.465 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-21T02:06:18.465 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-21T02:06:18.466 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T02:06:18.466 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:06:18.481 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T02:06:18.481 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-21T02:06:18.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:06:18.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.685+0000 7f6f9982e640 1 -- 172.21.15.88:0/3540429207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f7c003a30 msgr2=0x7f6f7c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.685+0000 7f6f91ffb640 1 -- 172.21.15.88:0/3540429207 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84006c20 con 0x7f6f7c003a30 2024-09-21T02:06:18.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.686+0000 7f6f9982e640 1 --2- 172.21.15.88:0/3540429207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f7c003a30 0x7f6f7c003ec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c260 tx=0x7f6f7c005830 comp rx=0 tx=0).stop 2024-09-21T02:06:18.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.686+0000 7f6f937fe640 1 -- 172.21.15.88:0/3540429207 reap_dead start 2024-09-21T02:06:18.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.689+0000 7f6f9982e640 1 -- 172.21.15.88:0/3540429207 shutdown_connections 2024-09-21T02:06:18.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.689+0000 7f6f9982e640 1 -- 172.21.15.88:0/3540429207 >> 172.21.15.88:0/3540429207 conn(0x7f6f94070c30 msgr2=0x7f6f94071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:18.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.692+0000 7f6f9982e640 1 -- 172.21.15.88:0/3540429207 shutdown_connections 2024-09-21T02:06:18.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.692+0000 7f6f9982e640 1 -- 172.21.15.88:0/3540429207 wait complete. 2024-09-21T02:06:18.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.698+0000 7f6f9982e640 1 Processor -- start 2024-09-21T02:06:18.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.698+0000 7f6f9982e640 1 -- start start 2024-09-21T02:06:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.702+0000 7f6f9982e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f94071040 0x7f6f941ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.702+0000 7f6f9982e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f94076710 0x7f6f94154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.702+0000 7f6f9982e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f941548c0 0x7f6f94153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.703+0000 7f6f937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f941548c0 0x7f6f94153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.703+0000 7f6f937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f941548c0 0x7f6f94153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38926/0 (socket says 172.21.15.88:38926) 2024-09-21T02:06:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.703+0000 7f6f937fe640 1 -- 172.21.15.88:0/804670140 learned_addr learned my addr 172.21.15.88:0/804670140 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.703+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94075a30 con 0x7f6f941548c0 2024-09-21T02:06:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.704+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f940758b0 con 0x7f6f94076710 2024-09-21T02:06:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.704+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6f94151df0 con 0x7f6f94071040 2024-09-21T02:06:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.705+0000 7f6f937fe640 1 -- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f94071040 msgr2=0x7f6f941ee8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T02:06:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.705+0000 7f6f937fe640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f94071040 0x7f6f941ee8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.705+0000 7f6f937fe640 1 -- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f94076710 msgr2=0x7f6f94154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.705+0000 7f6f937fe640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f94076710 0x7f6f94154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.705+0000 7f6f937fe640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f94154cd0 con 0x7f6f941548c0 2024-09-21T02:06:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.706+0000 7f6f937fe640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f941548c0 0x7f6f94153420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c005fd0 tx=0x7f6f8c00d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.706+0000 7f6f7bfff640 1 -- 172.21.15.88:0/804670140 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c01d070 con 0x7f6f941548c0 2024-09-21T02:06:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.707+0000 7f6f7bfff640 1 -- 172.21.15.88:0/804670140 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f8c003b70 con 0x7f6f941548c0 2024-09-21T02:06:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.707+0000 7f6f7bfff640 1 -- 172.21.15.88:0/804670140 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c004030 con 0x7f6f941548c0 2024-09-21T02:06:18.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.708+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f941ef210 con 0x7f6f941548c0 2024-09-21T02:06:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.708+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f941f26a0 con 0x7f6f941548c0 2024-09-21T02:06:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.710+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f941518b0 con 0x7f6f941548c0 2024-09-21T02:06:18.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.716+0000 7f6f7bfff640 1 -- 172.21.15.88:0/804670140 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6f8c020070 con 0x7f6f941548c0 2024-09-21T02:06:18.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.717+0000 7f6f7bfff640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6f80078c60 0x7f6f8007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:18.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.718+0000 7f6f7bfff640 1 -- 172.21.15.88:0/804670140 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6f8c09b9c0 con 0x7f6f941548c0 2024-09-21T02:06:18.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.721+0000 7f6f92ffd640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6f80078c60 0x7f6f8007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:18.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.722+0000 7f6f7bfff640 1 -- 172.21.15.88:0/804670140 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6f8c027080 con 0x7f6f941548c0 2024-09-21T02:06:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:18.739+0000 7f6f92ffd640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6f80078c60 0x7f6f8007b170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6f84033000 tx=0x7f6f84001a80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:19.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.217+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6f940cd840 con 0x7f6f941548c0 2024-09-21T02:06:19.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.217+0000 7f6f7bfff640 1 -- 172.21.15.88:0/804670140 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f6f8c063df0 con 0x7f6f941548c0 2024-09-21T02:06:19.219 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6f80078c60 msgr2=0x7f6f8007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6f80078c60 0x7f6f8007b170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6f84033000 tx=0x7f6f84001a80 comp rx=0 tx=0).stop 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f941548c0 msgr2=0x7f6f94153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f941548c0 0x7f6f94153420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c005fd0 tx=0x7f6f8c00d870 comp rx=0 tx=0).stop 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 shutdown_connections 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6f80078c60 0x7f6f8007b170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f941548c0 0x7f6f94153420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f94076710 0x7f6f94154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 --2- 172.21.15.88:0/804670140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f94071040 0x7f6f941ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:19.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.220+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 >> 172.21.15.88:0/804670140 conn(0x7f6f94070c30 msgr2=0x7f6f9414e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:19.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.221+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 shutdown_connections 2024-09-21T02:06:19.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:19.221+0000 7f6f9982e640 1 -- 172.21.15.88:0/804670140 wait complete. 2024-09-21T02:06:22.242 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:06:22.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.431+0000 7f754c1a8640 1 -- 172.21.15.88:0/2084913567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7530004160 msgr2=0x7f7530004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:22.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.431+0000 7f754c1a8640 1 --2- 172.21.15.88:0/2084913567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7530004160 0x7f7530004800 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7534001050 tx=0x7f7530005830 comp rx=0 tx=0).stop 2024-09-21T02:06:22.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.431+0000 7f754a745640 1 -- 172.21.15.88:0/2084913567 reap_dead start 2024-09-21T02:06:22.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.431+0000 7f754c1a8640 1 -- 172.21.15.88:0/2084913567 shutdown_connections 2024-09-21T02:06:22.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.431+0000 7f754c1a8640 1 -- 172.21.15.88:0/2084913567 >> 172.21.15.88:0/2084913567 conn(0x7f75440cd9f0 msgr2=0x7f75440cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:22.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.431+0000 7f754c1a8640 1 -- 172.21.15.88:0/2084913567 shutdown_connections 2024-09-21T02:06:22.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.431+0000 7f754c1a8640 1 -- 172.21.15.88:0/2084913567 wait complete. 2024-09-21T02:06:22.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.433+0000 7f754c1a8640 1 Processor -- start 2024-09-21T02:06:22.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.433+0000 7f754c1a8640 1 -- start start 2024-09-21T02:06:22.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f754c1a8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7530004160 0x7f754415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:22.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f754c1a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7544156e80 0x7f75441532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:22.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f754c1a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75441537e0 0x7f75441f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:22.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f754c1a8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f754415fd80 con 0x7f7544156e80 2024-09-21T02:06:22.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f754c1a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f754415c640 con 0x7f75441537e0 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f754c1a8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f754406dcd0 con 0x7f7530004160 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f7549f44640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7530004160 msgr2=0x7f754415dd80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f7549f44640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7530004160 0x7f754415dd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f7549743640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7544156e80 0x7f75441532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f7549743640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7544156e80 0x7f75441532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38954/0 (socket says 172.21.15.88:38954) 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f7549743640 1 -- 172.21.15.88:0/2689924585 learned_addr learned my addr 172.21.15.88:0/2689924585 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.435+0000 7f754a745640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75441537e0 0x7f75441f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f7549743640 1 -- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7530004160 msgr2=0x7f754415dd80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f7549743640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7530004160 0x7f754415dd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f7549743640 1 -- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75441537e0 msgr2=0x7f75441f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f7549743640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75441537e0 0x7f75441f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f7549743640 1 -- 172.21.15.88:0/2689924585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7544156740 con 0x7f7544156e80 2024-09-21T02:06:22.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f7549743640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7544156e80 0x7f75441532a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7534000c00 tx=0x7f7534032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:22.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f753a7fc640 1 -- 172.21.15.88:0/2689924585 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534037050 con 0x7f7544156e80 2024-09-21T02:06:22.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f753a7fc640 1 -- 172.21.15.88:0/2689924585 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7534043070 con 0x7f7544156e80 2024-09-21T02:06:22.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75441e6b20 con 0x7f7544156e80 2024-09-21T02:06:22.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f753a7fc640 1 -- 172.21.15.88:0/2689924585 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534005c50 con 0x7f7544156e80 2024-09-21T02:06:22.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.436+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75441e65a0 con 0x7f7544156e80 2024-09-21T02:06:22.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.438+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c005290 con 0x7f7544156e80 2024-09-21T02:06:22.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.439+0000 7f753a7fc640 1 -- 172.21.15.88:0/2689924585 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f753403cb40 con 0x7f7544156e80 2024-09-21T02:06:22.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.440+0000 7f753a7fc640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7528078b40 0x7f752807b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:22.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.440+0000 7f753a7fc640 1 -- 172.21.15.88:0/2689924585 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f75340bb7c0 con 0x7f7544156e80 2024-09-21T02:06:22.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.440+0000 7f7549f44640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7528078b40 0x7f752807b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:22.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.442+0000 7f7549f44640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7528078b40 0x7f752807b050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7540001220 tx=0x7f7540001550 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.446+0000 7f753a7fc640 1 -- 172.21.15.88:0/2689924585 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7534083540 con 0x7f7544156e80 2024-09-21T02:06:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.950+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f752c005850 con 0x7f7544156e80 2024-09-21T02:06:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.951+0000 7f753a7fc640 1 -- 172.21.15.88:0/2689924585 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f7534087f70 con 0x7f7544156e80 2024-09-21T02:06:22.952 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:22.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7528078b40 msgr2=0x7f752807b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:22.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7528078b40 0x7f752807b050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7540001220 tx=0x7f7540001550 comp rx=0 tx=0).stop 2024-09-21T02:06:22.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7544156e80 msgr2=0x7f75441532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:22.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7544156e80 0x7f75441532a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7534000c00 tx=0x7f7534032df0 comp rx=0 tx=0).stop 2024-09-21T02:06:22.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 shutdown_connections 2024-09-21T02:06:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7528078b40 0x7f752807b050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75441537e0 0x7f75441f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7544156e80 0x7f75441532a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 --2- 172.21.15.88:0/2689924585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7530004160 0x7f754415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 >> 172.21.15.88:0/2689924585 conn(0x7f75440cd9f0 msgr2=0x7f75440cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 shutdown_connections 2024-09-21T02:06:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:22.955+0000 7f754c1a8640 1 -- 172.21.15.88:0/2689924585 wait complete. 2024-09-21T02:06:25.975 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:06:26.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.177+0000 7f5f90189640 1 -- 172.21.15.88:0/873560009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 msgr2=0x7f5f881523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:26.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.177+0000 7f5f90189640 1 --2- 172.21.15.88:0/873560009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f881523c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5f84005fd0 tx=0x7f5f8402a670 comp rx=0 tx=0).stop 2024-09-21T02:06:26.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.177+0000 7f5f90189640 1 -- 172.21.15.88:0/873560009 shutdown_connections 2024-09-21T02:06:26.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.177+0000 7f5f90189640 1 --2- 172.21.15.88:0/873560009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f881523c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:26.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.177+0000 7f5f90189640 1 -- 172.21.15.88:0/873560009 >> 172.21.15.88:0/873560009 conn(0x7f5f880ca8d0 msgr2=0x7f5f880cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:26.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.178+0000 7f5f90189640 1 -- 172.21.15.88:0/873560009 shutdown_connections 2024-09-21T02:06:26.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.178+0000 7f5f90189640 1 -- 172.21.15.88:0/873560009 wait complete. 2024-09-21T02:06:26.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.181+0000 7f5f90189640 1 Processor -- start 2024-09-21T02:06:26.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.182+0000 7f5f90189640 1 -- start start 2024-09-21T02:06:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.185+0000 7f5f90189640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f8806cd60 0x7f5f881e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.185+0000 7f5f90189640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f880cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.185+0000 7f5f90189640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8815c190 0x7f5f881e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f8d724640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f880cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f8d724640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f880cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38976/0 (socket says 172.21.15.88:38976) 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f8d724640 1 -- 172.21.15.88:0/2746877517 learned_addr learned my addr 172.21.15.88:0/2746877517 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88073d00 con 0x7f5f88070730 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f8e726640 1 -- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8815c190 msgr2=0x7f5f881e64f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f8e726640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8815c190 0x7f5f881e64f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880c9280 con 0x7f5f8806cd60 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5f88075930 con 0x7f5f8815c190 2024-09-21T02:06:26.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.186+0000 7f5f8df25640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f8806cd60 0x7f5f881e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.187+0000 7f5f8d724640 1 -- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8815c190 msgr2=0x7f5f881e64f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.187+0000 7f5f8d724640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8815c190 0x7f5f881e64f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.187+0000 7f5f8d724640 1 -- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f8806cd60 msgr2=0x7f5f881e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.187+0000 7f5f8d724640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f8806cd60 0x7f5f881e5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.187+0000 7f5f8d724640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f881e5a50 con 0x7f5f88070730 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.187+0000 7f5f8d724640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f880cd3f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c340 tx=0x7f5f7800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.188+0000 7f5f7e7fc640 1 -- 172.21.15.88:0/2746877517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78017070 con 0x7f5f88070730 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.188+0000 7f5f7e7fc640 1 -- 172.21.15.88:0/2746877517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f780053c0 con 0x7f5f88070730 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.188+0000 7f5f7e7fc640 1 -- 172.21.15.88:0/2746877517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78003850 con 0x7f5f88070730 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.188+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f880c81d0 con 0x7f5f88070730 2024-09-21T02:06:26.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.188+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8814ce20 con 0x7f5f88070730 2024-09-21T02:06:26.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.189+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f880c8550 con 0x7f5f88070730 2024-09-21T02:06:26.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.190+0000 7f5f7e7fc640 1 -- 172.21.15.88:0/2746877517 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f7801a070 con 0x7f5f88070730 2024-09-21T02:06:26.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.191+0000 7f5f7e7fc640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:26.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.191+0000 7f5f7e7fc640 1 -- 172.21.15.88:0/2746877517 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5f78099f20 con 0x7f5f88070730 2024-09-21T02:06:26.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.192+0000 7f5f8df25640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:26.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.195+0000 7f5f8df25640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f84033000 tx=0x7f5f84004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:26.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.200+0000 7f5f7e7fc640 1 -- 172.21.15.88:0/2746877517 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f88155ff0 con 0x7f5f88070730 2024-09-21T02:06:26.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.545+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5f881e4140 con 0x7f5f88070730 2024-09-21T02:06:26.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.546+0000 7f5f7e7fc640 1 -- 172.21.15.88:0/2746877517 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f5f780624e0 con 0x7f5f88070730 2024-09-21T02:06:26.546 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5f6c078b90 msgr2=0x7f5f6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5f84033000 tx=0x7f5f84004730 comp rx=0 tx=0).stop 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 msgr2=0x7f5f880cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f880cd3f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c340 tx=0x7f5f7800c800 comp rx=0 tx=0).stop 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 shutdown_connections 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8815c190 0x7f5f881e64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f88070730 0x7f5f880cd3f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 --2- 172.21.15.88:0/2746877517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f8806cd60 0x7f5f881e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 >> 172.21.15.88:0/2746877517 conn(0x7f5f880ca8d0 msgr2=0x7f5f880cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 shutdown_connections 2024-09-21T02:06:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:26.549+0000 7f5f90189640 1 -- 172.21.15.88:0/2746877517 wait complete. 2024-09-21T02:06:27.522 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:27.522 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 rewriters = 1010807.50 kB/sec 2024-09-21T02:06:27.522 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 rewriters = 1010784.30 kB/sec 2024-09-21T02:06:27.522 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 1010807.50 kB/sec 2024-09-21T02:06:27.522 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 1010807.50 kB/sec 2024-09-21T02:06:27.523 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 1010807.50 kB/sec 2024-09-21T02:06:27.523 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 10485760.00 kB 2024-09-21T02:06:28.057 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:06:28.060 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:06:29.566 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:06:31.049 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:31.049 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 readers = 8013934.50 kB/sec 2024-09-21T02:06:31.050 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 readers = 8012004.89 kB/sec 2024-09-21T02:06:31.050 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 8013934.50 kB/sec 2024-09-21T02:06:31.050 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 8013934.50 kB/sec 2024-09-21T02:06:31.050 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 8013934.50 kB/sec 2024-09-21T02:06:31.050 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 10485760.00 kB 2024-09-21T02:06:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.059+0000 7fb3e8358640 1 -- 172.21.15.88:0/2590093366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c80039b0 msgr2=0x7fb3c8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.059+0000 7fb3e8358640 1 --2- 172.21.15.88:0/2590093366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c80039b0 0x7fb3c8003e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006400 tx=0x7fb3c8005830 comp rx=0 tx=0).stop 2024-09-21T02:06:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.060+0000 7fb3e8358640 1 -- 172.21.15.88:0/2590093366 shutdown_connections 2024-09-21T02:06:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.060+0000 7fb3e8358640 1 --2- 172.21.15.88:0/2590093366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3c8005350 0x7fb3c800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.060+0000 7fb3e8358640 1 --2- 172.21.15.88:0/2590093366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3c8004800 0x7fb3c8004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.060+0000 7fb3e8358640 1 --2- 172.21.15.88:0/2590093366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c80039b0 0x7fb3c8003e00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.060+0000 7fb3e8358640 1 -- 172.21.15.88:0/2590093366 >> 172.21.15.88:0/2590093366 conn(0x7fb3e00ce440 msgr2=0x7fb3e00ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.060+0000 7fb3e8358640 1 -- 172.21.15.88:0/2590093366 shutdown_connections 2024-09-21T02:06:33.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.060+0000 7fb3e8358640 1 -- 172.21.15.88:0/2590093366 wait complete. 2024-09-21T02:06:33.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.062+0000 7fb3e8358640 1 Processor -- start 2024-09-21T02:06:33.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.062+0000 7fb3e8358640 1 -- start start 2024-09-21T02:06:33.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e8358640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3c80039b0 0x7fb3e01ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e8358640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3c8004800 0x7fb3e015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e8358640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c8005350 0x7fb3e01446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e8358640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e00b3dd0 con 0x7fb3c8005350 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e8358640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e00b3c50 con 0x7fb3c80039b0 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e8358640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e0159180 con 0x7fb3c8004800 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e58f3640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3c8004800 msgr2=0x7fb3e015c2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e58f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3c8004800 0x7fb3e015c2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e68f5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c8005350 0x7fb3e01446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e68f5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c8005350 0x7fb3e01446f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49048/0 (socket says 172.21.15.88:49048) 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e68f5640 1 -- 172.21.15.88:0/1617428020 learned_addr learned my addr 172.21.15.88:0/1617428020 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.063+0000 7fb3e60f4640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3c80039b0 0x7fb3e01ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e68f5640 1 -- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3c8004800 msgr2=0x7fb3e015c2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e68f5640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3c8004800 0x7fb3e015c2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e68f5640 1 -- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3c80039b0 msgr2=0x7fb3e01ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e68f5640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3c80039b0 0x7fb3e01ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e68f5640 1 -- 172.21.15.88:0/1617428020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e01eecf0 con 0x7fb3c8005350 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e60f4640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3c80039b0 0x7fb3e01ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e68f5640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c8005350 0x7fb3e01446f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00c32a0 tx=0x7fb3d8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3deffd640 1 -- 172.21.15.88:0/1617428020 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8010920 con 0x7fb3c8005350 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3deffd640 1 -- 172.21.15.88:0/1617428020 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3d8010ac0 con 0x7fb3c8005350 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3deffd640 1 -- 172.21.15.88:0/1617428020 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8005020 con 0x7fb3c8005350 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e0159ec0 con 0x7fb3c8005350 2024-09-21T02:06:33.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.064+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e015a480 con 0x7fb3c8005350 2024-09-21T02:06:33.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.066+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e0159650 con 0x7fb3c8005350 2024-09-21T02:06:33.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.067+0000 7fb3deffd640 1 -- 172.21.15.88:0/1617428020 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb3d8023020 con 0x7fb3c8005350 2024-09-21T02:06:33.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.067+0000 7fb3deffd640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb3cc0793e0 0x7fb3cc07b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.067+0000 7fb3deffd640 1 -- 172.21.15.88:0/1617428020 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb3d809bbf0 con 0x7fb3c8005350 2024-09-21T02:06:33.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.067+0000 7fb3e60f4640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb3cc0793e0 0x7fb3cc07b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.068+0000 7fb3e60f4640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb3cc0793e0 0x7fb3cc07b8f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0036000 tx=0x7fb3d002d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.072+0000 7fb3deffd640 1 -- 172.21.15.88:0/1617428020 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb3d8064050 con 0x7fb3c8005350 2024-09-21T02:06:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.330+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb3e00c9110 con 0x7fb3c8005350 2024-09-21T02:06:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.330+0000 7fb3deffd640 1 -- 172.21.15.88:0/1617428020 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fb3d8063950 con 0x7fb3c8005350 2024-09-21T02:06:33.331 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":24,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb3cc0793e0 msgr2=0x7fb3cc07b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb3cc0793e0 0x7fb3cc07b8f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0036000 tx=0x7fb3d002d040 comp rx=0 tx=0).stop 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c8005350 msgr2=0x7fb3e01446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c8005350 0x7fb3e01446f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00c32a0 tx=0x7fb3d8011b90 comp rx=0 tx=0).stop 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 shutdown_connections 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb3cc0793e0 0x7fb3cc07b8f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3c8005350 0x7fb3e01446f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb3c8004800 0x7fb3e015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 --2- 172.21.15.88:0/1617428020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb3c80039b0 0x7fb3e01ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 >> 172.21.15.88:0/1617428020 conn(0x7fb3e00ce440 msgr2=0x7fb3e0151530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 shutdown_connections 2024-09-21T02:06:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.333+0000 7fb3e8358640 1 -- 172.21.15.88:0/1617428020 wait complete. 2024-09-21T02:06:33.347 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-21T02:06:33.347 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-21T02:06:33.347 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:06:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 -- 172.21.15.88:0/841007378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380cafc0 msgr2=0x7f013806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 --2- 172.21.15.88:0/841007378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380cafc0 0x7f013806cd60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f013806da20 tx=0x7f013002a820 comp rx=0 tx=0).stop 2024-09-21T02:06:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 -- 172.21.15.88:0/841007378 shutdown_connections 2024-09-21T02:06:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 --2- 172.21.15.88:0/841007378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380cafc0 0x7f013806cd60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 --2- 172.21.15.88:0/841007378 >> v2:172.21.15.195:3300/0 conn(0x7f01380b4c10 0x7f01380cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 -- 172.21.15.88:0/841007378 >> 172.21.15.88:0/841007378 conn(0x7f01380cda90 msgr2=0x7f01380cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 -- 172.21.15.88:0/841007378 shutdown_connections 2024-09-21T02:06:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.473+0000 7f013d94a640 1 -- 172.21.15.88:0/841007378 wait complete. 2024-09-21T02:06:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.475+0000 7f013d94a640 1 Processor -- start 2024-09-21T02:06:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.475+0000 7f013d94a640 1 -- start start 2024-09-21T02:06:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f013d94a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01380b4c10 0x7f01381f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f013d94a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01380cafc0 0x7f01381f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f013d94a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013815b980 0x7f01381f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f013d94a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01380b4410 con 0x7f013815b980 2024-09-21T02:06:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f013d94a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01380b4130 con 0x7f01380b4c10 2024-09-21T02:06:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f013d94a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f01380b46f0 con 0x7f01380cafc0 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f01377fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013815b980 0x7f01381f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f01367fc640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01380cafc0 msgr2=0x7f01381f5620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f01367fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01380cafc0 0x7f01381f5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f01377fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013815b980 0x7f01381f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49066/0 (socket says 172.21.15.88:49066) 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f01377fe640 1 -- 172.21.15.88:0/142774985 learned_addr learned my addr 172.21.15.88:0/142774985 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.476+0000 7f0136ffd640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01380b4c10 0x7f01381f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f01377fe640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01380cafc0 msgr2=0x7f01381f5620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f01377fe640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01380cafc0 0x7f01381f5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f01377fe640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01380b4c10 msgr2=0x7f01381f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f01377fe640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01380b4c10 0x7f01381f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f01377fe640 1 -- 172.21.15.88:0/142774985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01381f4af0 con 0x7f013815b980 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f01377fe640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013815b980 0x7f01381f5b60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0130009b30 tx=0x7f0130032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130037040 con 0x7f013815b980 2024-09-21T02:06:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01381f6170 con 0x7f013815b980 2024-09-21T02:06:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0130042070 con 0x7f013815b980 2024-09-21T02:06:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01300042a0 con 0x7f013815b980 2024-09-21T02:06:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.477+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01381e4200 con 0x7f013815b980 2024-09-21T02:06:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.479+0000 7f013d94a640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380c4230 0x7f01381e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.479+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01380cb990 con 0x7f01380c4230 2024-09-21T02:06:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.479+0000 7f0136ffd640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380c4230 0x7f01381e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.479+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0130004440 con 0x7f013815b980 2024-09-21T02:06:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.479+0000 7f0136ffd640 1 -- 172.21.15.88:0/142774985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01380c4920 con 0x7f01380c4230 2024-09-21T02:06:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.479+0000 7f0136ffd640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380c4230 0x7f01381e44e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f012c00f540 tx=0x7f012c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0124078b70 0x7f012407b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f01300bbcb0 con 0x7f013815b980 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c018020 con 0x7f01380c4230 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f01367fc640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0124078b70 0x7f012407b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f012c014680 con 0x7f01380c4230 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380c4230 msgr2=0x7f01381e44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380c4230 0x7f01381e44e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f012c00f540 tx=0x7f012c00fa00 comp rx=0 tx=0).stop 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380c4230 msgr2=0x7f01381e44e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.480+0000 7f011ffff640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01380c4230 0x7f01381e44e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.481+0000 7f01367fc640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0124078b70 0x7f012407b080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01381efa80 tx=0x7f0128005b90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.486+0000 7f013d94a640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0138158e70 0x7f0138153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.486+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01381e6fd0 con 0x7f0138158e70 2024-09-21T02:06:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.487+0000 7f0136ffd640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0138158e70 0x7f0138153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.487+0000 7f0136ffd640 1 -- 172.21.15.88:0/142774985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f01380ca570 con 0x7f0138158e70 2024-09-21T02:06:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.487+0000 7f0136ffd640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0138158e70 0x7f0138153340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f012c019000 tx=0x7f012c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.487+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c018070 con 0x7f0138158e70 2024-09-21T02:06:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5041 (secure 0 0 0) 0x7f012c014c70 con 0x7f0138158e70 2024-09-21T02:06:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0138158e70 msgr2=0x7f0138153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f011ffff640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0138158e70 0x7f0138153340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f012c019000 tx=0x7f012c000ed0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f011ffff640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0138158e70 msgr2=0x7f0138153340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f011ffff640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0138158e70 0x7f0138153340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.488 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:33.488 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 24, 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 181203, 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: 1 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:33.489 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:33.490 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:33.491 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:33.492 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.493 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.494 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.495 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:33.496 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.497 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.498 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.499 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.500 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:33.501 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:33.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0124078b70 msgr2=0x7f012407b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f0124078b70 0x7f012407b080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01381efa80 tx=0x7f0128005b90 comp rx=0 tx=0).stop 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013815b980 msgr2=0x7f01381f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 --2- 172.21.15.88:0/142774985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013815b980 0x7f01381f5b60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0130009b30 tx=0x7f0130032d10 comp rx=0 tx=0).stop 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f01377fe640 1 -- 172.21.15.88:0/142774985 reap_dead start 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f01377fe640 1 -- 172.21.15.88:0/142774985 reap_dead start 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 shutdown_connections 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 >> 172.21.15.88:0/142774985 conn(0x7f01380cda90 msgr2=0x7f0138155340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 shutdown_connections 2024-09-21T02:06:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.488+0000 7f013d94a640 1 -- 172.21.15.88:0/142774985 wait complete. 2024-09-21T02:06:33.503 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-21T02:06:33.504 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-21T02:06:33.504 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:06:33.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 -- 172.21.15.88:0/2181866259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c91d0 msgr2=0x7f6c7806de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2181866259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c91d0 0x7f6c7806de70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005fd0 tx=0x7f6c7002a8e0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 -- 172.21.15.88:0/2181866259 shutdown_connections 2024-09-21T02:06:33.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2181866259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c91d0 0x7f6c7806de70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2181866259 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c780c7af0 0x7f6c780c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 -- 172.21.15.88:0/2181866259 >> 172.21.15.88:0/2181866259 conn(0x7f6c780720b0 msgr2=0x7f6c780cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 -- 172.21.15.88:0/2181866259 shutdown_connections 2024-09-21T02:06:33.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.630+0000 7f6c7e825640 1 -- 172.21.15.88:0/2181866259 wait complete. 2024-09-21T02:06:33.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.632+0000 7f6c7e825640 1 Processor -- start 2024-09-21T02:06:33.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.632+0000 7f6c7e825640 1 -- start start 2024-09-21T02:06:33.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7e825640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c2e60 0x7f6c7814bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7e825640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c780c7af0 0x7f6c781576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7e825640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c780c91d0 0x7f6c781f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7e825640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78159190 con 0x7f6c780c7af0 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7e825640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78159480 con 0x7f6c780c2e60 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7e825640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7806f1f0 con 0x7f6c780c91d0 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7cdc2640 1 -- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c780c91d0 msgr2=0x7f6c781f2340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.195:3301/0 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c780c7af0 0x7f6c781576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c7cdc2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c780c91d0 0x7f6c781f2340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c780c7af0 0x7f6c781576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49100/0 (socket says 172.21.15.88:49100) 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.633+0000 7f6c777fe640 1 -- 172.21.15.88:0/2377170291 learned_addr learned my addr 172.21.15.88:0/2377170291 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c77fff640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c2e60 0x7f6c7814bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c777fe640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c780c91d0 msgr2=0x7f6c781f2340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:33.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c777fe640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c780c91d0 0x7f6c781f2340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c777fe640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c2e60 msgr2=0x7f6c7814bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c777fe640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c2e60 0x7f6c7814bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c777fe640 1 -- 172.21.15.88:0/2377170291 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c78157c10 con 0x7f6c780c7af0 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c777fe640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c780c7af0 0x7f6c781576d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b4d0 tx=0x7f6c6800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68014070 con 0x7f6c780c7af0 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c68010960 con 0x7f6c780c7af0 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c781f2db0 con 0x7f6c780c7af0 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68010db0 con 0x7f6c780c7af0 2024-09-21T02:06:33.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.634+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c780c6490 con 0x7f6c780c7af0 2024-09-21T02:06:33.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.636+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c6690 0x7f6c7814a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.636+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781580e0 con 0x7f6c780c6690 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.636+0000 7f6c77fff640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c6690 0x7f6c7814a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.636+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c68017070 con 0x7f6c780c7af0 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.636+0000 7f6c77fff640 1 -- 172.21.15.88:0/2377170291 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c780c8c90 con 0x7f6c780c6690 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c77fff640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c6690 0x7f6c7814a850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000f770 tx=0x7f6c6000fc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c58078b40 0x7f6c5807b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6c6809aa50 con 0x7f6c780c7af0 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c7cdc2640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c58078b40 0x7f6c5807b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60018020 con 0x7f6c780c6690 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6c6000d750 con 0x7f6c780c6690 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c6690 msgr2=0x7f6c7814a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c6690 0x7f6c7814a850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000f770 tx=0x7f6c6000fc30 comp rx=0 tx=0).stop 2024-09-21T02:06:33.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c6690 msgr2=0x7f6c7814a850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:33.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c74ff9640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c780c6690 0x7f6c7814a850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.637+0000 7f6c7cdc2640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c58078b40 0x7f6c5807b050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c70034000 tx=0x7f6c70004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.643+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c781f2880 0x7f6c7814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:33.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.644+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78074160 con 0x7f6c781f2880 2024-09-21T02:06:33.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.644+0000 7f6c77fff640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c781f2880 0x7f6c7814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:33.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.644+0000 7f6c77fff640 1 -- 172.21.15.88:0/2377170291 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6c7814f880 con 0x7f6c781f2880 2024-09-21T02:06:33.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c77fff640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c781f2880 0x7f6c7814ad90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c60019000 tx=0x7f6c60001440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:33.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60018040 con 0x7f6c781f2880 2024-09-21T02:06:33.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5015 (secure 0 0 0) 0x7f6c6001f220 con 0x7f6c781f2880 2024-09-21T02:06:33.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c781f2880 msgr2=0x7f6c7814ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c74ff9640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c781f2880 0x7f6c7814ad90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c60019000 tx=0x7f6c60001440 comp rx=0 tx=0).stop 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c74ff9640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c781f2880 msgr2=0x7f6c7814ad90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c74ff9640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c781f2880 0x7f6c7814ad90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 24, 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 181318, 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:06:33.646 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: 1 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 0, 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:33.647 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:33.648 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:06:33.649 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:06:33.650 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.651 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:33.652 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.653 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:06:33.654 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.655 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.656 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:06:33.657 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:06:33.658 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:06:33.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c58078b40 msgr2=0x7f6c5807b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f6c58078b40 0x7f6c5807b050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c70034000 tx=0x7f6c70004ab0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c7cdc2640 1 -- 172.21.15.88:0/2377170291 reap_dead start 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c780c7af0 msgr2=0x7f6c781576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c780c7af0 0x7f6c781576d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b4d0 tx=0x7f6c6800b990 comp rx=0 tx=0).stop 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.645+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 shutdown_connections 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.646+0000 7f6c7e825640 1 --2- 172.21.15.88:0/2377170291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c780c7af0 0x7f6c781576d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.646+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 >> 172.21.15.88:0/2377170291 conn(0x7f6c780720b0 msgr2=0x7f6c78074680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.646+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 shutdown_connections 2024-09-21T02:06:33.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:33.646+0000 7f6c7e825640 1 -- 172.21.15.88:0/2377170291 wait complete. 2024-09-21T02:06:33.660 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-21T02:06:33.661 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-21T02:06:34.473 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:34.473 INFO:tasks.workunit.client.0.smithi088.stdout: Children see throughput for 1 re-readers = 7834238.00 kB/sec 2024-09-21T02:06:34.473 INFO:tasks.workunit.client.0.smithi088.stdout: Parent sees throughput for 1 re-readers = 7833137.53 kB/sec 2024-09-21T02:06:34.473 INFO:tasks.workunit.client.0.smithi088.stdout: Min throughput per process = 7834238.00 kB/sec 2024-09-21T02:06:34.473 INFO:tasks.workunit.client.0.smithi088.stdout: Max throughput per process = 7834238.00 kB/sec 2024-09-21T02:06:34.473 INFO:tasks.workunit.client.0.smithi088.stdout: Avg throughput per process = 7834238.00 kB/sec 2024-09-21T02:06:34.473 INFO:tasks.workunit.client.0.smithi088.stdout: Min xfer = 10485760.00 kB 2024-09-21T02:06:41.838 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:41.839 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:41.839 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-09-21T02:06:41.839 INFO:tasks.workunit.client.0.smithi088.stdout:iozone test complete. 2024-09-21T02:06:41.840 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T02:06:41.840 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T02:06:41.873 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-21T02:06:41.874 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T02:06:42.215 DEBUG:teuthology.parallel:result is None 2024-09-21T02:06:42.215 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T02:06:42.265 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T02:06:42.265 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-21T02:06:42.274 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-21T02:06:53.661 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-21T02:06:53.661 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T02:06:53.662 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T02:06:53.664 INFO:tasks.ceph.mon.c:Started 2024-09-21T02:06:53.665 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-21T02:06:53.665 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:06:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.790+0000 7ff00a139640 1 -- 172.21.15.88:0/3658925977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefec004800 msgr2=0x7fefec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.790+0000 7ff00a139640 1 --2- 172.21.15.88:0/3658925977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefec004800 0x7fefec004c10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feff8006fc0 tx=0x7fefec005830 comp rx=0 tx=0).stop 2024-09-21T02:06:53.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.790+0000 7ff00a139640 1 -- 172.21.15.88:0/3658925977 shutdown_connections 2024-09-21T02:06:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.790+0000 7ff00a139640 1 --2- 172.21.15.88:0/3658925977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefec004800 0x7fefec004c10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.790+0000 7ff00a139640 1 -- 172.21.15.88:0/3658925977 >> 172.21.15.88:0/3658925977 conn(0x7ff0040cd9f0 msgr2=0x7ff0040cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.790+0000 7ff00a139640 1 -- 172.21.15.88:0/3658925977 shutdown_connections 2024-09-21T02:06:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.790+0000 7ff00a139640 1 -- 172.21.15.88:0/3658925977 wait complete. 2024-09-21T02:06:53.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.792+0000 7ff00a139640 1 Processor -- start 2024-09-21T02:06:53.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.792+0000 7ff00a139640 1 -- start start 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff00a139640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefec004800 0x7ff00415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff00a139640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 0x7ff0040c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff00a139640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0040c97c0 0x7ff0040c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff00a139640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff00406dcd0 con 0x7ff0040c97c0 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff00a139640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff00415c640 con 0x7fefec004800 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff00a139640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ff00406de50 con 0x7ff0041f5c20 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff003fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0040c97c0 0x7ff0040c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff003fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0040c97c0 0x7ff0040c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37128/0 (socket says 172.21.15.88:37128) 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff003fff640 1 -- 172.21.15.88:0/965561229 learned_addr learned my addr 172.21.15.88:0/965561229 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff0037fe640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefec004800 0x7ff00415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff002ffd640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 0x7ff0040c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.793+0000 7ff002ffd640 1 -- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 msgr2=0x7ff0040c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff002ffd640 1 -- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 msgr2=0x7ff0040c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff002ffd640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 0x7ff0040c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff002ffd640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 0x7ff0040c6330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff003fff640 1 -- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 msgr2=0x7ff0040c6330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff003fff640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 0x7ff0040c6330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff003fff640 1 -- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefec004800 msgr2=0x7ff00415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff003fff640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefec004800 0x7ff00415d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff003fff640 1 -- 172.21.15.88:0/965561229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0040c6870 con 0x7ff0040c97c0 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff003fff640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0040c97c0 0x7ff0040c9bd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff0040c8830 tx=0x7feffc006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7fefebfff640 1 -- 172.21.15.88:0/965561229 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc015070 con 0x7ff0040c97c0 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7fefebfff640 1 -- 172.21.15.88:0/965561229 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feffc011920 con 0x7ff0040c97c0 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7fefebfff640 1 -- 172.21.15.88:0/965561229 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc011d70 con 0x7ff0040c97c0 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0040c6a80 con 0x7ff0040c97c0 2024-09-21T02:06:53.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.794+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00415e610 con 0x7ff0040c97c0 2024-09-21T02:06:53.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.795+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00415fb30 con 0x7ff0040c97c0 2024-09-21T02:06:53.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.796+0000 7fefebfff640 1 -- 172.21.15.88:0/965561229 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feffc018070 con 0x7ff0040c97c0 2024-09-21T02:06:53.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.796+0000 7fefebfff640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7feff0078bc0 0x7feff007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:06:53.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.797+0000 7fefebfff640 1 -- 172.21.15.88:0/965561229 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7feffc09b930 con 0x7ff0040c97c0 2024-09-21T02:06:53.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.797+0000 7ff0037fe640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7feff0078bc0 0x7feff007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:06:53.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.797+0000 7ff0037fe640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7feff0078bc0 0x7feff007b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff00408e7f0 tx=0x7feff4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:06:53.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:53.801+0000 7fefebfff640 1 -- 172.21.15.88:0/965561229 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feffc0637c0 con 0x7ff0040c97c0 2024-09-21T02:06:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:54.045+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff00415e080 con 0x7ff0040c97c0 2024-09-21T02:06:58.114 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:06:58.117 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:06:58.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.983+0000 7fefebfff640 1 -- 172.21.15.88:0/965561229 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7feffc0681f0 con 0x7ff0040c97c0 2024-09-21T02:06:58.984 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":28,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:06:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7feff0078bc0 msgr2=0x7feff007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7feff0078bc0 0x7feff007b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff00408e7f0 tx=0x7feff4005f90 comp rx=0 tx=0).stop 2024-09-21T02:06:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0040c97c0 msgr2=0x7ff0040c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:06:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0040c97c0 0x7ff0040c9bd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff0040c8830 tx=0x7feffc006430 comp rx=0 tx=0).stop 2024-09-21T02:06:58.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 shutdown_connections 2024-09-21T02:06:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7feff0078bc0 0x7feff007b0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0040c97c0 0x7ff0040c9bd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0041f5c20 0x7ff0040c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 --2- 172.21.15.88:0/965561229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefec004800 0x7ff00415d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:06:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 >> 172.21.15.88:0/965561229 conn(0x7ff0040cd9f0 msgr2=0x7ff0040b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:06:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 shutdown_connections 2024-09-21T02:06:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:06:58.986+0000 7ff00a139640 1 -- 172.21.15.88:0/965561229 wait complete. 2024-09-21T02:07:02.003 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:07:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 -- 172.21.15.88:0/3225855833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd80c71e0 msgr2=0x7fefd80ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 --2- 172.21.15.88:0/3225855833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd80c71e0 0x7fefd80ca250 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009830 tx=0x7fefc002a510 comp rx=0 tx=0).stop 2024-09-21T02:07:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 -- 172.21.15.88:0/3225855833 shutdown_connections 2024-09-21T02:07:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 --2- 172.21.15.88:0/3225855833 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd80ca790 0x7fefd8158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 --2- 172.21.15.88:0/3225855833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd80c71e0 0x7fefd80ca250 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 -- 172.21.15.88:0/3225855833 >> 172.21.15.88:0/3225855833 conn(0x7fefd806d180 msgr2=0x7fefd806d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 -- 172.21.15.88:0/3225855833 shutdown_connections 2024-09-21T02:07:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.131+0000 7fefde2bd640 1 -- 172.21.15.88:0/3225855833 wait complete. 2024-09-21T02:07:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.132+0000 7fefde2bd640 1 Processor -- start 2024-09-21T02:07:02.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.133+0000 7fefde2bd640 1 -- start start 2024-09-21T02:07:02.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefde2bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd80c71e0 0x7fefd81f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefde2bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd80ca790 0x7fefd8149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefde2bd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd81527f0 0x7fefd8146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefde2bd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fefd80c6260 con 0x7fefd81527f0 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefde2bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8073760 con 0x7fefd80c71e0 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefde2bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fefd8152c00 con 0x7fefd80ca790 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd81527f0 0x7fefd8146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd81527f0 0x7fefd8146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38588/0 (socket says 172.21.15.88:38588) 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 -- 172.21.15.88:0/2483135971 learned_addr learned my addr 172.21.15.88:0/2483135971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefd77fe640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd80ca790 0x7fefd8149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefd7fff640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd80c71e0 0x7fefd81f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 -- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd80ca790 msgr2=0x7fefd8149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd80ca790 0x7fefd8149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 -- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd80c71e0 msgr2=0x7fefd81f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd80c71e0 0x7fefd81f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.134+0000 7fefdc85a640 1 -- 172.21.15.88:0/2483135971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd81f2ac0 con 0x7fefd81527f0 2024-09-21T02:07:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.135+0000 7fefdc85a640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd81527f0 0x7fefd8146bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e960 tx=0x7fefd000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.135+0000 7fefd4ff9640 1 -- 172.21.15.88:0/2483135971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0004d20 con 0x7fefd81527f0 2024-09-21T02:07:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.135+0000 7fefd4ff9640 1 -- 172.21.15.88:0/2483135971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefd000cac0 con 0x7fefd81527f0 2024-09-21T02:07:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.135+0000 7fefd4ff9640 1 -- 172.21.15.88:0/2483135971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0005020 con 0x7fefd81527f0 2024-09-21T02:07:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.135+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd8151360 con 0x7fefd81527f0 2024-09-21T02:07:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.135+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd8151bf0 con 0x7fefd81527f0 2024-09-21T02:07:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.136+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd80c75f0 con 0x7fefd81527f0 2024-09-21T02:07:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.136+0000 7fefd4ff9640 1 -- 172.21.15.88:0/2483135971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefd00051c0 con 0x7fefd81527f0 2024-09-21T02:07:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.137+0000 7fefd4ff9640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fefb8078bc0 0x7fefb807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.137+0000 7fefd4ff9640 1 -- 172.21.15.88:0/2483135971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fefd009b620 con 0x7fefd81527f0 2024-09-21T02:07:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.137+0000 7fefd7fff640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fefb8078bc0 0x7fefb807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.138+0000 7fefd7fff640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fefb8078bc0 0x7fefb807b0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefc8001540 tx=0x7fefc8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.142+0000 7fefd4ff9640 1 -- 172.21.15.88:0/2483135971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fefd0016030 con 0x7fefd81527f0 2024-09-21T02:07:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.389+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fefd806b0d0 con 0x7fefd81527f0 2024-09-21T02:07:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.390+0000 7fefd4ff9640 1 -- 172.21.15.88:0/2483135971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7fefd0063a70 con 0x7fefd81527f0 2024-09-21T02:07:02.391 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:07:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fefb8078bc0 msgr2=0x7fefb807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fefb8078bc0 0x7fefb807b0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fefc8001540 tx=0x7fefc8009040 comp rx=0 tx=0).stop 2024-09-21T02:07:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd81527f0 msgr2=0x7fefd8146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd81527f0 0x7fefd8146bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e960 tx=0x7fefd000ee20 comp rx=0 tx=0).stop 2024-09-21T02:07:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 shutdown_connections 2024-09-21T02:07:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fefb8078bc0 0x7fefb807b0d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefd81527f0 0x7fefd8146bc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefd80ca790 0x7fefd8149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 --2- 172.21.15.88:0/2483135971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefd80c71e0 0x7fefd81f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 >> 172.21.15.88:0/2483135971 conn(0x7fefd806d180 msgr2=0x7fefd81567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 shutdown_connections 2024-09-21T02:07:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.392+0000 7fefde2bd640 1 -- 172.21.15.88:0/2483135971 wait complete. 2024-09-21T02:07:02.404 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-21T02:07:02.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-21T02:07:02.404 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-21T02:07:02.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.517+0000 7fb6441b0640 1 -- 172.21.15.88:0/3412494595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb624004800 msgr2=0x7fb624004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.517+0000 7fb6441b0640 1 --2- 172.21.15.88:0/3412494595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb624004800 0x7fb624004c10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb6300016c0 tx=0x7fb624005830 comp rx=0 tx=0).stop 2024-09-21T02:07:02.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.517+0000 7fb64274d640 1 -- 172.21.15.88:0/3412494595 reap_dead start 2024-09-21T02:07:02.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.517+0000 7fb6441b0640 1 -- 172.21.15.88:0/3412494595 shutdown_connections 2024-09-21T02:07:02.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.518+0000 7fb6441b0640 1 -- 172.21.15.88:0/3412494595 >> 172.21.15.88:0/3412494595 conn(0x7fb63c06de70 msgr2=0x7fb63c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:02.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.518+0000 7fb6441b0640 1 -- 172.21.15.88:0/3412494595 shutdown_connections 2024-09-21T02:07:02.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.518+0000 7fb6441b0640 1 -- 172.21.15.88:0/3412494595 wait complete. 2024-09-21T02:07:02.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.519+0000 7fb6441b0640 1 Processor -- start 2024-09-21T02:07:02.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.519+0000 7fb6441b0640 1 -- start start 2024-09-21T02:07:02.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.520+0000 7fb6441b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb624004800 0x7fb63c1f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.520+0000 7fb6441b0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c0cd010 0x7fb63c1e3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.520+0000 7fb6441b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 0x7fb63c1e3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.520+0000 7fb6441b0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c0c9c50 con 0x7fb63c0cd010 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.520+0000 7fb6441b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c0c9970 con 0x7fb624004800 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.520+0000 7fb6441b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fb63c072190 con 0x7fb63c0cd420 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c0cd010 0x7fb63c1e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c0cd010 0x7fb63c1e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38600/0 (socket says 172.21.15.88:38600) 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 -- 172.21.15.88:0/3443206419 learned_addr learned my addr 172.21.15.88:0/3443206419 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb641f4c640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb624004800 0x7fb63c1f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64274d640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 0x7fb63c1e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64274d640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 msgr2=0x7fb63c1e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64274d640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 msgr2=0x7fb63c1e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64274d640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 0x7fb63c1e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64274d640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 0x7fb63c1e3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:07:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 msgr2=0x7fb63c1e3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63c0cd420 0x7fb63c1e3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb624004800 msgr2=0x7fb63c1f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb624004800 0x7fb63c1f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 -- 172.21.15.88:0/3443206419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c1efbe0 con 0x7fb63c0cd010 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb64174b640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c0cd010 0x7fb63c1e3200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb63002a930 tx=0x7fb630006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630038070 con 0x7fb63c0cd010 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb630003af0 con 0x7fb63c0cd010 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63003d460 con 0x7fb63c0cd010 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c14f840 con 0x7fb63c0cd010 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.521+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c14efd0 con 0x7fb63c0cd010 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.522+0000 7fb6441b0640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151170 0x7fb63c151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.522+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1f0be0 con 0x7fb63c151170 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.522+0000 7fb641f4c640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151170 0x7fb63c151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb641f4c640 1 -- 172.21.15.88:0/3443206419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb63c074660 con 0x7fb63c151170 2024-09-21T02:07:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb641f4c640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151170 0x7fb63c151580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb62c001050 tx=0x7fb62c0066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0129d0 con 0x7fb63c151170 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb62c012b70 con 0x7fb63c151170 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151170 msgr2=0x7fb63c151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb63a7fc640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151170 0x7fb63c151580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb62c001050 tx=0x7fb62c0066b0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151170 msgr2=0x7fb63c151580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb63a7fc640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151170 0x7fb63c151580 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb62c001050 tx=0x7fb62c0066b0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.523+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb630030070 con 0x7fb63c0cd010 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.524+0000 7fb63a7fc640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb628078640 0x7fb62807ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.524+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb6300ba600 con 0x7fb63c0cd010 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.524+0000 7fb641f4c640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb628078640 0x7fb62807ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.525+0000 7fb641f4c640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb628078640 0x7fb62807ab50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000c00 tx=0x7fb62c00f000 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.529+0000 7fb6441b0640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151ac0 0x7fb63c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.529+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c1f3500 con 0x7fb63c151ac0 2024-09-21T02:07:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.529+0000 7fb64274d640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151ac0 0x7fb63c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb64274d640 1 -- 172.21.15.88:0/3443206419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb63c0c3370 con 0x7fb63c151ac0 2024-09-21T02:07:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb64274d640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151ac0 0x7fb63c0cbcb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb63c159590 tx=0x7fb634010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634004b20 con 0x7fb63c151ac0 2024-09-21T02:07:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5052 (secure 0 0 0) 0x7fb634004b20 con 0x7fb63c151ac0 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151ac0 msgr2=0x7fb63c0cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb63a7fc640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151ac0 0x7fb63c0cbcb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb63c159590 tx=0x7fb634010740 comp rx=0 tx=0).stop 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb63a7fc640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151ac0 msgr2=0x7fb63c0cbcb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb63a7fc640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c151ac0 0x7fb63c0cbcb0 secure :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb63c159590 tx=0x7fb634010740 comp rx=0 tx=0).stop 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "leader", 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 30, 2024-09-21T02:07:02.532 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 210246, 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 2, 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:02.533 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:02.534 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:07:02.535 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:07:02.536 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:07:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.538 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:02.539 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.540 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:02.541 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:02.542 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.543 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.544 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 3 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": [ 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:07:02.545 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb628078640 msgr2=0x7fb62807ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb6441b0640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fb628078640 0x7fb62807ab50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000c00 tx=0x7fb62c00f000 comp rx=0 tx=0).stop 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c0cd010 msgr2=0x7fb63c1e3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb64274d640 1 -- 172.21.15.88:0/3443206419 reap_dead start 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb6441b0640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c0cd010 0x7fb63c1e3200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb63002a930 tx=0x7fb630006360 comp rx=0 tx=0).stop 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 shutdown_connections 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb6441b0640 1 --2- 172.21.15.88:0/3443206419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb63c0cd010 0x7fb63c1e3200 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.530+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 >> 172.21.15.88:0/3443206419 conn(0x7fb63c06de70 msgr2=0x7fb63c148700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.531+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 shutdown_connections 2024-09-21T02:07:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.531+0000 7fb6441b0640 1 -- 172.21.15.88:0/3443206419 wait complete. 2024-09-21T02:07:02.547 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:07:02.547 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-21T02:07:02.548 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-21T02:07:02.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 -- 172.21.15.88:0/2005935929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3658153a70 msgr2=0x7f36580cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 --2- 172.21.15.88:0/2005935929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3658153a70 0x7f36580cef30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3640009860 tx=0x7f364002a420 comp rx=0 tx=0).stop 2024-09-21T02:07:02.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 -- 172.21.15.88:0/2005935929 shutdown_connections 2024-09-21T02:07:02.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 --2- 172.21.15.88:0/2005935929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3658153a70 0x7f36580cef30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 --2- 172.21.15.88:0/2005935929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3658068d00 0x7f365815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 -- 172.21.15.88:0/2005935929 >> 172.21.15.88:0/2005935929 conn(0x7f3658070960 msgr2=0x7f365806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:02.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 -- 172.21.15.88:0/2005935929 shutdown_connections 2024-09-21T02:07:02.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.660+0000 7f365ce30640 1 -- 172.21.15.88:0/2005935929 wait complete. 2024-09-21T02:07:02.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.662+0000 7f365ce30640 1 Processor -- start 2024-09-21T02:07:02.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.662+0000 7f365ce30640 1 -- start start 2024-09-21T02:07:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f365ce30640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3658068d00 0x7f36581e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f365ce30640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36580693a0 0x7f36581f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f365ce30640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3658153a70 0x7f36580ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f365ce30640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3658159ed0 con 0x7f36580693a0 2024-09-21T02:07:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f365ce30640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3658159990 con 0x7f3658153a70 2024-09-21T02:07:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f365ce30640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f365815a6d0 con 0x7f3658068d00 2024-09-21T02:07:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36580693a0 0x7f36581f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36580693a0 0x7f36581f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38622/0 (socket says 172.21.15.88:38622) 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 -- 172.21.15.88:0/3437173141 learned_addr learned my addr 172.21.15.88:0/3437173141 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f3656d9d640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3658153a70 0x7f36580ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f365659c640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3658068d00 0x7f36581e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3658068d00 msgr2=0x7f36581e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3658068d00 0x7f36581e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3658153a70 msgr2=0x7f36580ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3658153a70 0x7f36580ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.663+0000 7f364dd9b640 1 -- 172.21.15.88:0/3437173141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36581f3a30 con 0x7f36580693a0 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.664+0000 7f364dd9b640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36580693a0 0x7f36581f34f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f364002aa10 tx=0x7f3640033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.664+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640005c90 con 0x7f36580693a0 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.664+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3640031e50 con 0x7f36580693a0 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.664+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36580c83e0 con 0x7f36580693a0 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.664+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640005260 con 0x7f36580693a0 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.664+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36581e68a0 con 0x7f36580693a0 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.665+0000 7f365ce30640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36580c7a10 0x7f365814b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.665+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36581f3f00 con 0x7f36580c7a10 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.665+0000 7f365659c640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36580c7a10 0x7f365814b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.665+0000 7f365659c640 1 -- 172.21.15.88:0/3437173141 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36580c86a0 con 0x7f36580c7a10 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.665+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3640005400 con 0x7f36580693a0 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.665+0000 7f365659c640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36580c7a10 0x7f365814b830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f364800e680 tx=0x7f364800eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3638078bc0 0x7f363807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f36400bbf80 con 0x7f36580693a0 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f3656d9d640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3638078bc0 0x7f363807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648016070 con 0x7f36580c7a10 2024-09-21T02:07:02.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3648004230 con 0x7f36580c7a10 2024-09-21T02:07:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36580c7a10 msgr2=0x7f365814b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36580c7a10 0x7f365814b830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f364800e680 tx=0x7f364800eb40 comp rx=0 tx=0).stop 2024-09-21T02:07:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36580c7a10 msgr2=0x7f365814b830 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:07:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.666+0000 7f364f7fe640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36580c7a10 0x7f365814b830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.667+0000 7f3656d9d640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3638078bc0 0x7f363807b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3650004a30 tx=0x7f3650009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.672+0000 7f365ce30640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f365814bd70 0x7f36580ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.672+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f365806a3d0 con 0x7f365814bd70 2024-09-21T02:07:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.672+0000 7f365659c640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f365814bd70 0x7f36580ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f365659c640 1 -- 172.21.15.88:0/3437173141 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f365814c180 con 0x7f365814bd70 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f365659c640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f365814bd70 0x7f36580ccda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f364800e680 tx=0x7f36480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648011470 con 0x7f365814bd70 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 <== mon.1 v2:172.21.15.195:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5026 (secure 0 0 0) 0x7f36480103d0 con 0x7f365814bd70 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f365814bd70 msgr2=0x7f36580ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f364f7fe640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f365814bd70 0x7f36580ccda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f364800e680 tx=0x7f36480046c0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f364f7fe640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f365814bd70 msgr2=0x7f36580ccda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.673+0000 7f364f7fe640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f365814bd70 0x7f36580ccda0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f364800e680 tx=0x7f36480046c0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.675 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 30, 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 210346, 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 2, 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:07:02.676 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:02.677 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:02.678 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:07:02.679 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:02.680 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:07:02.681 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:02.682 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.683 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:07:02.684 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.686 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": [ 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": [ 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 2 2024-09-21T02:07:02.687 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x2f018fb87aa4aafe", 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "luminous", 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:02.688 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:07:02.689 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3638078bc0 msgr2=0x7f363807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f3638078bc0 0x7f363807b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3650004a30 tx=0x7f3650009040 comp rx=0 tx=0).stop 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36580693a0 msgr2=0x7f36581f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 --2- 172.21.15.88:0/3437173141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36580693a0 0x7f36581f34f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f364002aa10 tx=0x7f3640033c90 comp rx=0 tx=0).stop 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f3656d9d640 1 -- 172.21.15.88:0/3437173141 reap_dead start 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f3656d9d640 1 -- 172.21.15.88:0/3437173141 reap_dead start 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 shutdown_connections 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 >> 172.21.15.88:0/3437173141 conn(0x7f3658070960 msgr2=0x7f3658150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 shutdown_connections 2024-09-21T02:07:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.674+0000 7f365ce30640 1 -- 172.21.15.88:0/3437173141 wait complete. 2024-09-21T02:07:02.691 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:07:02.691 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-21T02:07:02.691 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-21T02:07:02.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 -- 172.21.15.88:0/388202463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f4158810 msgr2=0x7f64f4151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 --2- 172.21.15.88:0/388202463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f4158810 0x7f64f4151ba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f64f40725e0 tx=0x7f64f002a5d0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 -- 172.21.15.88:0/388202463 shutdown_connections 2024-09-21T02:07:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 --2- 172.21.15.88:0/388202463 >> v2:172.21.15.195:3300/0 conn(0x7f64f40c8690 0x7f64f406e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 --2- 172.21.15.88:0/388202463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f4158810 0x7f64f4151ba0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 -- 172.21.15.88:0/388202463 >> 172.21.15.88:0/388202463 conn(0x7f64f406d8f0 msgr2=0x7f64f40ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 -- 172.21.15.88:0/388202463 shutdown_connections 2024-09-21T02:07:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.802+0000 7f64fc54a640 1 -- 172.21.15.88:0/388202463 wait complete. 2024-09-21T02:07:02.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.804+0000 7f64fc54a640 1 Processor -- start 2024-09-21T02:07:02.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.804+0000 7f64fc54a640 1 -- start start 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64fc54a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f40c8690 0x7f64f40b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64fc54a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f41520e0 0x7f64f4157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64fc54a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 0x7f64f414ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64fc54a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64f406f2b0 con 0x7f64f41520e0 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64fc54a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4158c20 con 0x7f64f40c8690 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64fc54a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f64f406f430 con 0x7f64f4158810 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64f9ae5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f41520e0 0x7f64f4157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64f9ae5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f41520e0 0x7f64f4157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38648/0 (socket says 172.21.15.88:38648) 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64f9ae5640 1 -- 172.21.15.88:0/1285029590 learned_addr learned my addr 172.21.15.88:0/1285029590 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.805+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f40c8690 0x7f64f40b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64faae7640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 0x7f64f414ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64faae7640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 msgr2=0x7f64f414ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64faae7640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 msgr2=0x7f64f414ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64faae7640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 0x7f64f414ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64f9ae5640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 msgr2=0x7f64f414ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64faae7640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 0x7f64f414ab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64f9ae5640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f4158810 0x7f64f414ab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64f9ae5640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f40c8690 msgr2=0x7f64f40b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64f9ae5640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64f40c8690 0x7f64f40b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64f9ae5640 1 -- 172.21.15.88:0/1285029590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f4157d40 con 0x7f64f41520e0 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64f9ae5640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f41520e0 0x7f64f4157140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c3a0 tx=0x7f64e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4017070 con 0x7f64f41520e0 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64e40035f0 con 0x7f64f41520e0 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f40cb980 con 0x7f64f41520e0 2024-09-21T02:07:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4003a40 con 0x7f64f41520e0 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.806+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f40cc390 con 0x7f64f41520e0 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.807+0000 7f64fc54a640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.807+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f64f4157b10 con 0x7f64f40cb220 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.807+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.807+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.807+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.807+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.808+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T02:07:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.808+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64e401a070 con 0x7f64f41520e0 2024-09-21T02:07:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.808+0000 7f64eaffd640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f64d8078bc0 0x7f64d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.808+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f64e409a0c0 con 0x7f64f41520e0 2024-09-21T02:07:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.808+0000 7f64faae7640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f64d8078bc0 0x7f64d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:02.809+0000 7f64faae7640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f64d8078bc0 0x7f64d807b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005c40 tx=0x7f64ec005bd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:03.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.009+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.009+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-21T02:07:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.009+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:07:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.009+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:07:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.009+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T02:07:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.411+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.411+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-21T02:07:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.411+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T02:07:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.411+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T02:07:03.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:03.411+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-21T02:07:04.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.213+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:04.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.214+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64f4151ba0 con 0x7f64f40cb220 2024-09-21T02:07:04.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.214+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f64f40b33d0 tx=0x7f64f0003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:04.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.214+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0003c50 con 0x7f64f40cb220 2024-09-21T02:07:04.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.215+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f64f0003c50 con 0x7f64f40cb220 2024-09-21T02:07:04.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.215+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.215+0000 7f64eaffd640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f64f40b33d0 tx=0x7f64f0003890 comp rx=0 tx=0).stop 2024-09-21T02:07:04.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.215+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 msgr2=0x7f64f414b040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:07:04.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.215+0000 7f64eaffd640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cb220 0x7f64f414b040 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.221+0000 7f64fc54a640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cc560 0x7f64f4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:04.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.221+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f64f414b580 con 0x7f64f40cc560 2024-09-21T02:07:04.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.222+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cc560 0x7f64f4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.222+0000 7f64fa2e6640 1 -- 172.21.15.88:0/1285029590 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64f40c3bc0 con 0x7f64f40cc560 2024-09-21T02:07:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.222+0000 7f64fa2e6640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cc560 0x7f64f4155c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f64f40cb630 tx=0x7f64f002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0005020 con 0x7f64f40cc560 2024-09-21T02:07:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 <== mon.2 v2:172.21.15.195:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4548 (secure 0 0 0) 0x7f64f0005020 con 0x7f64f40cc560 2024-09-21T02:07:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cc560 msgr2=0x7f64f4155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64eaffd640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cc560 0x7f64f4155c30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f64f40cb630 tx=0x7f64f002a5d0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64eaffd640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cc560 msgr2=0x7f64f4155c30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-21T02:07:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64eaffd640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64f40cc560 0x7f64f4155c30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.226 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:04.226 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:07:04.226 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:07:04.226 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "peon", 2024-09-21T02:07:04.226 INFO:teuthology.orchestra.run.smithi088.stdout: "election_epoch": 30, 2024-09-21T02:07:04.226 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 10458, 2024-09-21T02:07:04.227 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum": [ 2024-09-21T02:07:04.227 INFO:teuthology.orchestra.run.smithi088.stdout: 0, 2024-09-21T02:07:04.227 INFO:teuthology.orchestra.run.smithi088.stdout: 1, 2024-09-21T02:07:04.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2 2024-09-21T02:07:04.227 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:04.227 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_age": 4, 2024-09-21T02:07:04.227 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:07:04.228 INFO:teuthology.orchestra.run.smithi088.stdout: "required_con": "2450521708007866372", 2024-09-21T02:07:04.228 INFO:teuthology.orchestra.run.smithi088.stdout: "required_mon": [ 2024-09-21T02:07:04.228 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:04.228 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:04.228 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:04.228 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:04.228 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:04.229 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_con": "4540719139924082687", 2024-09-21T02:07:04.230 INFO:teuthology.orchestra.run.smithi088.stdout: "quorum_mon": [ 2024-09-21T02:07:04.230 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:04.230 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:04.230 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:04.230 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:04.230 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:04.230 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "outside_quorum": [], 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "extra_probe_peers": [], 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "sync_provider": [], 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "monmap": { 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 1, 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "fsid": "0ee3c2c3-8406-48b8-a5fb-ec622d976411", 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "modified": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout: "created": "2024-09-21T02:02:59.122435Z", 2024-09-21T02:07:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release": 19, 2024-09-21T02:07:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: "min_mon_release_name": "squid", 2024-09-21T02:07:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: "election_strategy": 1, 2024-09-21T02:07:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: "disallowed_leaders: ": "", 2024-09-21T02:07:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false, 2024-09-21T02:07:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: "tiebreaker_mon": "", 2024-09-21T02:07:04.233 INFO:teuthology.orchestra.run.smithi088.stdout: "removed_ranks: ": "", 2024-09-21T02:07:04.234 INFO:teuthology.orchestra.run.smithi088.stdout: "features": { 2024-09-21T02:07:04.234 INFO:teuthology.orchestra.run.smithi088.stdout: "persistent": [ 2024-09-21T02:07:04.234 INFO:teuthology.orchestra.run.smithi088.stdout: "kraken", 2024-09-21T02:07:04.234 INFO:teuthology.orchestra.run.smithi088.stdout: "luminous", 2024-09-21T02:07:04.234 INFO:teuthology.orchestra.run.smithi088.stdout: "mimic", 2024-09-21T02:07:04.234 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap-prune", 2024-09-21T02:07:04.234 INFO:teuthology.orchestra.run.smithi088.stdout: "nautilus", 2024-09-21T02:07:04.235 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus", 2024-09-21T02:07:04.235 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific", 2024-09-21T02:07:04.235 INFO:teuthology.orchestra.run.smithi088.stdout: "elector-pinging", 2024-09-21T02:07:04.235 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy", 2024-09-21T02:07:04.235 INFO:teuthology.orchestra.run.smithi088.stdout: "reef", 2024-09-21T02:07:04.235 INFO:teuthology.orchestra.run.smithi088.stdout: "squid" 2024-09-21T02:07:04.236 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:04.236 INFO:teuthology.orchestra.run.smithi088.stdout: "optional": [] 2024-09-21T02:07:04.236 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.236 INFO:teuthology.orchestra.run.smithi088.stdout: "mons": [ 2024-09-21T02:07:04.236 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.236 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 0, 2024-09-21T02:07:04.236 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "a", 2024-09-21T02:07:04.237 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:04.237 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:04.237 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.237 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:04.237 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:3300", 2024-09-21T02:07:04.237 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:04.237 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.238 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.238 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:04.238 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789", 2024-09-21T02:07:04.238 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:04.238 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:04.238 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:04.238 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.239 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.88:6789/0", 2024-09-21T02:07:04.239 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.88:6789/0", 2024-09-21T02:07:04.239 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:04.239 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:04.239 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:04.239 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.239 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.240 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 1, 2024-09-21T02:07:04.240 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "b", 2024-09-21T02:07:04.240 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:04.240 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:04.240 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.240 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:04.240 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3300", 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789", 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:04.241 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:04.242 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.242 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6789/0", 2024-09-21T02:07:04.242 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6789/0", 2024-09-21T02:07:04.242 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:04.242 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:04.242 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:04.242 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.243 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.243 INFO:teuthology.orchestra.run.smithi088.stdout: "rank": 2, 2024-09-21T02:07:04.243 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "c", 2024-09-21T02:07:04.243 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addrs": { 2024-09-21T02:07:04.243 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T02:07:04.243 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.243 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:3301", 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790", 2024-09-21T02:07:04.244 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 0 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.195:6790/0", 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: "public_addr": "172.21.15.195:6790/0", 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: "priority": 0, 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: "weight": 0, 2024-09-21T02:07:04.245 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_location": "{}" 2024-09-21T02:07:04.246 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:04.246 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:04.246 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.246 INFO:teuthology.orchestra.run.smithi088.stdout: "feature_map": { 2024-09-21T02:07:04.246 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": [ 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: "client": [ 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: "features": "0x3f03dffffffdffff", 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: "release": "squid", 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 1 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T02:07:04.247 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T02:07:04.248 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T02:07:04.248 INFO:teuthology.orchestra.run.smithi088.stdout: "stretch_mode": false 2024-09-21T02:07:04.248 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f64d8078bc0 msgr2=0x7f64d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f64d8078bc0 0x7f64d807b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005c40 tx=0x7f64ec005bd0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f41520e0 msgr2=0x7f64f4157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 --2- 172.21.15.88:0/1285029590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64f41520e0 0x7f64f4157140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c3a0 tx=0x7f64e400c860 comp rx=0 tx=0).stop 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64faae7640 1 -- 172.21.15.88:0/1285029590 reap_dead start 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64faae7640 1 -- 172.21.15.88:0/1285029590 reap_dead start 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 shutdown_connections 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 >> 172.21.15.88:0/1285029590 conn(0x7f64f406d8f0 msgr2=0x7f64f414f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 shutdown_connections 2024-09-21T02:07:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.223+0000 7f64fc54a640 1 -- 172.21.15.88:0/1285029590 wait complete. 2024-09-21T02:07:04.250 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-21T02:07:04.250 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-21T02:07:04.250 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-21T02:07:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.370+0000 7f86038d5640 1 -- 172.21.15.88:0/3413181188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 msgr2=0x7f85fc06ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.370+0000 7f86038d5640 1 --2- 172.21.15.88:0/3413181188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc06ec30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85f4005fd0 tx=0x7f85f402a820 comp rx=0 tx=0).stop 2024-09-21T02:07:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.370+0000 7f86038d5640 1 -- 172.21.15.88:0/3413181188 shutdown_connections 2024-09-21T02:07:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.370+0000 7f86038d5640 1 --2- 172.21.15.88:0/3413181188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc06ec30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.370+0000 7f86038d5640 1 --2- 172.21.15.88:0/3413181188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85fc06de70 0x7f85fc158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.370+0000 7f86038d5640 1 -- 172.21.15.88:0/3413181188 >> 172.21.15.88:0/3413181188 conn(0x7f85fc0729d0 msgr2=0x7f85fc072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.371+0000 7f86038d5640 1 -- 172.21.15.88:0/3413181188 shutdown_connections 2024-09-21T02:07:04.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.371+0000 7f86038d5640 1 -- 172.21.15.88:0/3413181188 wait complete. 2024-09-21T02:07:04.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.372+0000 7f86038d5640 1 Processor -- start 2024-09-21T02:07:04.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.372+0000 7f86038d5640 1 -- start start 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f86038d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85fc06de70 0x7f85fc14f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f86038d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f86038d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85fc0c5a20 0x7f85fc1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f86038d5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc15c7f0 con 0x7f85fc06e820 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f86038d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc0c5f20 con 0x7f85fc06de70 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f86038d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc15c420 con 0x7f85fc0c5a20 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38662/0 (socket says 172.21.15.88:38662) 2024-09-21T02:07:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 -- 172.21.15.88:0/384573796 learned_addr learned my addr 172.21.15.88:0/384573796 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8601e72640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85fc0c5a20 0x7f85fc1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8601671640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85fc06de70 0x7f85fc14f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 -- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85fc0c5a20 msgr2=0x7f85fc1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85fc0c5a20 0x7f85fc1512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 -- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85fc06de70 msgr2=0x7f85fc14f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85fc06de70 0x7f85fc14f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 -- 172.21.15.88:0/384573796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc14ea70 con 0x7f85fc06e820 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f8600e70640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc150d70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c970 tx=0x7f85ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f85f1ffb640 1 -- 172.21.15.88:0/384573796 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec007bb0 con 0x7f85fc06e820 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f85f1ffb640 1 -- 172.21.15.88:0/384573796 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85ec007d50 con 0x7f85fc06e820 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc14fcb0 con 0x7f85fc06e820 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.374+0000 7f85f1ffb640 1 -- 172.21.15.88:0/384573796 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec005280 con 0x7f85fc06e820 2024-09-21T02:07:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.375+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc1f2870 con 0x7f85fc06e820 2024-09-21T02:07:04.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.376+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85fc072360 con 0x7f85fc06e820 2024-09-21T02:07:04.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.376+0000 7f85f1ffb640 1 -- 172.21.15.88:0/384573796 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f85ec005420 con 0x7f85fc06e820 2024-09-21T02:07:04.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.377+0000 7f85f1ffb640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f85e0078b90 0x7f85e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:04.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.377+0000 7f85f1ffb640 1 -- 172.21.15.88:0/384573796 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f85ec09b0f0 con 0x7f85fc06e820 2024-09-21T02:07:04.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.377+0000 7f8601671640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f85e0078b90 0x7f85e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:04.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.377+0000 7f8601671640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f85e0078b90 0x7f85e007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000f60 tx=0x7f85f8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:04.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.381+0000 7f85f1ffb640 1 -- 172.21.15.88:0/384573796 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f85ec014030 con 0x7f85fc06e820 2024-09-21T02:07:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.632+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f85fc0cac80 con 0x7f85fc06e820 2024-09-21T02:07:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.632+0000 7f85f1ffb640 1 -- 172.21.15.88:0/384573796 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f85ec062ef0 con 0x7f85fc06e820 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f85e0078b90 msgr2=0x7f85e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f85e0078b90 0x7f85e007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000f60 tx=0x7f85f8009040 comp rx=0 tx=0).stop 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 msgr2=0x7f85fc150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc150d70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c970 tx=0x7f85ec00ce30 comp rx=0 tx=0).stop 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 shutdown_connections 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f85e0078b90 0x7f85e007b0a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85fc0c5a20 0x7f85fc1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85fc06e820 0x7f85fc150d70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 --2- 172.21.15.88:0/384573796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85fc06de70 0x7f85fc14f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:04.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.634+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 >> 172.21.15.88:0/384573796 conn(0x7f85fc0729d0 msgr2=0x7f85fc154760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.635+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 shutdown_connections 2024-09-21T02:07:04.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:04.635+0000 7f86038d5640 1 -- 172.21.15.88:0/384573796 wait complete. 2024-09-21T02:07:04.647 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-21T02:07:14.648 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:07:14.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 -- 172.21.15.88:0/1721734154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c4f10 msgr2=0x7fe4bc070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/1721734154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c4f10 0x7fe4bc070fd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc074970 tx=0x7fe4b402aac0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 -- 172.21.15.88:0/1721734154 shutdown_connections 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/1721734154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c4f10 0x7fe4bc070fd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/1721734154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4bc0c8a00 0x7fe4bc0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/1721734154 >> v2:172.21.15.195:3300/0 conn(0x7fe4bc0c3fc0 0x7fe4bc156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 -- 172.21.15.88:0/1721734154 >> 172.21.15.88:0/1721734154 conn(0x7fe4bc0729d0 msgr2=0x7fe4bc072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 -- 172.21.15.88:0/1721734154 shutdown_connections 2024-09-21T02:07:14.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.772+0000 7fe4c30d3640 1 -- 172.21.15.88:0/1721734154 wait complete. 2024-09-21T02:07:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.775+0000 7fe4c30d3640 1 Processor -- start 2024-09-21T02:07:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.775+0000 7fe4c30d3640 1 -- start start 2024-09-21T02:07:14.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4bc0c3fc0 0x7fe4bc1e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4bc0c4f10 0x7fe4bc1e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c8a00 0x7fe4bc1f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc15a5b0 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc070ab0 con 0x7fe4bc0c4f10 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe4bc0c8f30 con 0x7fe4bc0c3fc0 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c8a00 0x7fe4bc1f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c8a00 0x7fe4bc1f05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39510/0 (socket says 172.21.15.88:39510) 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 -- 172.21.15.88:0/4033244241 learned_addr learned my addr 172.21.15.88:0/4033244241 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c0e6f640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4bc0c3fc0 0x7fe4bc1e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4bbfff640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4bc0c4f10 0x7fe4bc1e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 -- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4bc0c3fc0 msgr2=0x7fe4bc1e3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4bc0c3fc0 0x7fe4bc1e3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 -- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4bc0c4f10 msgr2=0x7fe4bc1e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4bc0c4f10 0x7fe4bc1e0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 -- 172.21.15.88:0/4033244241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc1dfb40 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c1670640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c8a00 0x7fe4bc1f05a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe4b402b0b0 tx=0x7fe4b4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4b97fa640 1 -- 172.21.15.88:0/4033244241 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4004250 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4b97fa640 1 -- 172.21.15.88:0/4033244241 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4b40043f0 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4bc1dfe20 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4b97fa640 1 -- 172.21.15.88:0/4033244241 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4004840 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.776+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4bc1e20b0 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.777+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4bc0c5fc0 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.778+0000 7fe4b97fa640 1 -- 172.21.15.88:0/4033244241 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4b4049430 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.778+0000 7fe4b97fa640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe4a8078bc0 0x7fe4a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.778+0000 7fe4b97fa640 1 -- 172.21.15.88:0/4033244241 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe4b40bba00 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.779+0000 7fe4c0e6f640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe4a8078bc0 0x7fe4a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.779+0000 7fe4c0e6f640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe4a8078bc0 0x7fe4a807b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00bb50 tx=0x7fe4ac00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:14.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.783+0000 7fe4b97fa640 1 -- 172.21.15.88:0/4033244241 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4bc0c5fc0 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.994+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4bc156df0 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.995+0000 7fe4b97fa640 1 -- 172.21.15.88:0/4033244241 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6908 (secure 0 0 0) 0x7fe4b4032030 con 0x7fe4bc0c8a00 2024-09-21T02:07:14.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:14.996 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":13,"btime":"2024-09-21T02:04:16:141092+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:16.141080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4398,4399,4401]},"id":1}]} 2024-09-21T02:07:14.997 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 13 2024-09-21T02:07:14.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe4a8078bc0 msgr2=0x7fe4a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:14.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe4a8078bc0 0x7fe4a807b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00bb50 tx=0x7fe4ac00b2e0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c8a00 msgr2=0x7fe4bc1f05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c8a00 0x7fe4bc1f05a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe4b402b0b0 tx=0x7fe4b4003ce0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 shutdown_connections 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe4a8078bc0 0x7fe4a807b0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4bc0c8a00 0x7fe4bc1f05a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4bc0c4f10 0x7fe4bc1e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 --2- 172.21.15.88:0/4033244241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe4bc0c3fc0 0x7fe4bc1e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 >> 172.21.15.88:0/4033244241 conn(0x7fe4bc0729d0 msgr2=0x7fe4bc1587c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 shutdown_connections 2024-09-21T02:07:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:14.997+0000 7fe4c30d3640 1 -- 172.21.15.88:0/4033244241 wait complete. 2024-09-21T02:07:15.010 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-21T02:07:15.010 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 -- 172.21.15.88:0/3202605492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0c97e0 msgr2=0x7f213c162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 --2- 172.21.15.88:0/3202605492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0c97e0 0x7f213c162f10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2134005fd0 tx=0x7f213402ab20 comp rx=0 tx=0).stop 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 -- 172.21.15.88:0/3202605492 shutdown_connections 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 --2- 172.21.15.88:0/3202605492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0c97e0 0x7f213c162f10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 --2- 172.21.15.88:0/3202605492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f213c1526a0 0x7f213c0c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 -- 172.21.15.88:0/3202605492 >> 172.21.15.88:0/3202605492 conn(0x7f213c0ca8d0 msgr2=0x7f213c0cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 -- 172.21.15.88:0/3202605492 shutdown_connections 2024-09-21T02:07:15.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.121+0000 7f214437b640 1 -- 172.21.15.88:0/3202605492 wait complete. 2024-09-21T02:07:15.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.123+0000 7f214437b640 1 Processor -- start 2024-09-21T02:07:15.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.123+0000 7f214437b640 1 -- start start 2024-09-21T02:07:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f214437b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f213c0c97e0 0x7f213c14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f214437b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f213c1526a0 0x7f213c150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f214437b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c158f60 0x7f213c1477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f214437b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f213c0cb770 con 0x7f213c158f60 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f214437b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f213c158a40 con 0x7f213c0c97e0 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f214437b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f213c0759e0 con 0x7f213c1526a0 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f2142918640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c158f60 0x7f213c1477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f2142918640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c158f60 0x7f213c1477d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39532/0 (socket says 172.21.15.88:39532) 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.124+0000 7f2142918640 1 -- 172.21.15.88:0/2785882419 learned_addr learned my addr 172.21.15.88:0/2785882419 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2142117640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f213c0c97e0 0x7f213c14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2141916640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f213c1526a0 0x7f213c150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2142918640 1 -- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f213c1526a0 msgr2=0x7f213c150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2142918640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f213c1526a0 0x7f213c150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2142918640 1 -- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f213c0c97e0 msgr2=0x7f213c14d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2142918640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f213c0c97e0 0x7f213c14d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2142918640 1 -- 172.21.15.88:0/2785882419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c14c550 con 0x7f213c158f60 2024-09-21T02:07:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2142918640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c158f60 0x7f213c1477d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2134009bf0 tx=0x7f2134003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2132ffd640 1 -- 172.21.15.88:0/2785882419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134044070 con 0x7f213c158f60 2024-09-21T02:07:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2132ffd640 1 -- 172.21.15.88:0/2785882419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2134004250 con 0x7f213c158f60 2024-09-21T02:07:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c06f760 con 0x7f213c158f60 2024-09-21T02:07:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f2132ffd640 1 -- 172.21.15.88:0/2785882419 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134004660 con 0x7f213c158f60 2024-09-21T02:07:15.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.125+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c14d1b0 con 0x7f213c158f60 2024-09-21T02:07:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.126+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c159370 con 0x7f213c158f60 2024-09-21T02:07:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.127+0000 7f2132ffd640 1 -- 172.21.15.88:0/2785882419 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2134036400 con 0x7f213c158f60 2024-09-21T02:07:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.127+0000 7f2132ffd640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2120078bc0 0x7f212007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.127+0000 7f2132ffd640 1 -- 172.21.15.88:0/2785882419 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f21340bb570 con 0x7f213c158f60 2024-09-21T02:07:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.128+0000 7f2142117640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2120078bc0 0x7f212007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.128+0000 7f2142117640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2120078bc0 0x7f212007b0d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2138005fd0 tx=0x7f2138005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:15.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.133+0000 7f2132ffd640 1 -- 172.21.15.88:0/2785882419 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2134032030 con 0x7f213c158f60 2024-09-21T02:07:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.375+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f213c0cdae0 con 0x7f213c158f60 2024-09-21T02:07:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.375+0000 7f2132ffd640 1 -- 172.21.15.88:0/2785882419 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f2134083a50 con 0x7f213c158f60 2024-09-21T02:07:15.376 INFO:teuthology.orchestra.run.smithi088.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","modified":"2024-09-21T02:02:59.122435Z","created":"2024-09-21T02:02:59.122435Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3301","nonce":0},{"type":"v1","addr":"172.21.15.195:6790","nonce":0}]},"addr":"172.21.15.195:6790/0","public_addr":"172.21.15.195:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-21T02:07:15.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2120078bc0 msgr2=0x7f212007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2120078bc0 0x7f212007b0d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2138005fd0 tx=0x7f2138005850 comp rx=0 tx=0).stop 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c158f60 msgr2=0x7f213c1477d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c158f60 0x7f213c1477d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2134009bf0 tx=0x7f2134003ce0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 shutdown_connections 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2120078bc0 0x7f212007b0d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c158f60 0x7f213c1477d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f213c1526a0 0x7f213c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 --2- 172.21.15.88:0/2785882419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f213c0c97e0 0x7f213c14d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 >> 172.21.15.88:0/2785882419 conn(0x7f213c0ca8d0 msgr2=0x7f213c0cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 shutdown_connections 2024-09-21T02:07:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.377+0000 7f214437b640 1 -- 172.21.15.88:0/2785882419 wait complete. 2024-09-21T02:07:15.390 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-21T02:07:15.390 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T02:07:15.405 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T02:07:15.405 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:07:15.405 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:07:15.421 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T02:07:15.421 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:07:15.477 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:07:15.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.631+0000 7f79fdb0b640 1 -- 172.21.15.88:0/3595918480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0004d40 msgr2=0x7f79e0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.631+0000 7f79fdb0b640 1 --2- 172.21.15.88:0/3595918480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0004d40 0x7f79e0005150 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009ed0 tx=0x7f79e0005830 comp rx=0 tx=0).stop 2024-09-21T02:07:15.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.631+0000 7f79f7fff640 1 -- 172.21.15.88:0/3595918480 reap_dead start 2024-09-21T02:07:15.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.631+0000 7f79fdb0b640 1 -- 172.21.15.88:0/3595918480 shutdown_connections 2024-09-21T02:07:15.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.631+0000 7f79fdb0b640 1 -- 172.21.15.88:0/3595918480 >> 172.21.15.88:0/3595918480 conn(0x7f79f806d8f0 msgr2=0x7f79f80ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:15.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.631+0000 7f79fdb0b640 1 -- 172.21.15.88:0/3595918480 shutdown_connections 2024-09-21T02:07:15.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.631+0000 7f79fdb0b640 1 -- 172.21.15.88:0/3595918480 wait complete. 2024-09-21T02:07:15.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.633+0000 7f79fdb0b640 1 Processor -- start 2024-09-21T02:07:15.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.633+0000 7f79fdb0b640 1 -- start start 2024-09-21T02:07:15.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79fdb0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f81488a0 0x7f79f80b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79fdb0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f81f2060 0x7f79f80b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79fdb0b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f813f8e0 0x7f79f806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79fdb0b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8075940 con 0x7f79f813f8e0 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79fdb0b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f79f80c9710 con 0x7f79f81488a0 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79fdb0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f79f81594d0 con 0x7f79f81f2060 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79f7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f813f8e0 0x7f79f806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79f7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f813f8e0 0x7f79f806e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39560/0 (socket says 172.21.15.88:39560) 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.634+0000 7f79f7fff640 1 -- 172.21.15.88:0/715958844 learned_addr learned my addr 172.21.15.88:0/715958844 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f6ffd640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f81f2060 0x7f79f80b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f77fe640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f81488a0 0x7f79f80b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f7fff640 1 -- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f81f2060 msgr2=0x7f79f80b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f7fff640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f81f2060 0x7f79f80b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f7fff640 1 -- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f81488a0 msgr2=0x7f79f80b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f7fff640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f81488a0 0x7f79f80b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f7fff640 1 -- 172.21.15.88:0/715958844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f81562b0 con 0x7f79f813f8e0 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f77fe640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f81488a0 0x7f79f80b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79f7fff640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f813f8e0 0x7f79f806e5b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f79f002aed0 tx=0x7f79f0033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79dffff640 1 -- 172.21.15.88:0/715958844 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0042070 con 0x7f79f813f8e0 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79dffff640 1 -- 172.21.15.88:0/715958844 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79f0003970 con 0x7f79f813f8e0 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f80b2430 con 0x7f79f813f8e0 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79dffff640 1 -- 172.21.15.88:0/715958844 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0038040 con 0x7f79f813f8e0 2024-09-21T02:07:15.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.635+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f815b470 con 0x7f79f813f8e0 2024-09-21T02:07:15.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.636+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f80c1ec0 con 0x7f79f813f8e0 2024-09-21T02:07:15.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.637+0000 7f79dffff640 1 -- 172.21.15.88:0/715958844 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f79f0004030 con 0x7f79f813f8e0 2024-09-21T02:07:15.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.637+0000 7f79dffff640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f79e4078b90 0x7f79e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:15.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.638+0000 7f79dffff640 1 -- 172.21.15.88:0/715958844 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f79f00baeb0 con 0x7f79f813f8e0 2024-09-21T02:07:15.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.638+0000 7f79f77fe640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f79e4078b90 0x7f79e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:15.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.638+0000 7f79f77fe640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f79e4078b90 0x7f79e407b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f79e8001860 tx=0x7f79e8001510 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:15.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.642+0000 7f79dffff640 1 -- 172.21.15.88:0/715958844 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f79f0082e00 con 0x7f79f813f8e0 2024-09-21T02:07:15.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.833+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f79f8075510 con 0x7f79f813f8e0 2024-09-21T02:07:15.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.833+0000 7f79dffff640 1 -- 172.21.15.88:0/715958844 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f79f0087830 con 0x7f79f813f8e0 2024-09-21T02:07:15.833 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f79e4078b90 msgr2=0x7f79e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f79e4078b90 0x7f79e407b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f79e8001860 tx=0x7f79e8001510 comp rx=0 tx=0).stop 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f813f8e0 msgr2=0x7f79f806e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f813f8e0 0x7f79f806e5b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f79f002aed0 tx=0x7f79f0033c10 comp rx=0 tx=0).stop 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 shutdown_connections 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f79e4078b90 0x7f79e407b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79f813f8e0 0x7f79f806e5b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79f81f2060 0x7f79f80b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 --2- 172.21.15.88:0/715958844 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79f81488a0 0x7f79f80b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.835+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 >> 172.21.15.88:0/715958844 conn(0x7f79f806d8f0 msgr2=0x7f79f814aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:15.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.836+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 shutdown_connections 2024-09-21T02:07:15.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:15.836+0000 7f79fdb0b640 1 -- 172.21.15.88:0/715958844 wait complete. 2024-09-21T02:07:15.847 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T02:07:15.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:07:15.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T02:07:15.887 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:07:15.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:07:15.887 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:07:15.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:07:15.959 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T02:07:16.014 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:07:16.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:07:16.014 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:07:16.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:07:16.080 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T02:07:16.171 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T02:07:16.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:07:16.180 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:07:16.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.293+0000 7fc4c4fad640 1 -- 172.21.15.88:0/1157176217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 msgr2=0x7fc4a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.293+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/1157176217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4a800b9c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00744e0 tx=0x7fc4a8005830 comp rx=0 tx=0).stop 2024-09-21T02:07:16.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.293+0000 7fc4c4fad640 1 -- 172.21.15.88:0/1157176217 shutdown_connections 2024-09-21T02:07:16.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.293+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/1157176217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4a800b9c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.293+0000 7fc4c4fad640 1 -- 172.21.15.88:0/1157176217 >> 172.21.15.88:0/1157176217 conn(0x7fc4c0070c30 msgr2=0x7fc4c0071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:16.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.293+0000 7fc4c4fad640 1 -- 172.21.15.88:0/1157176217 shutdown_connections 2024-09-21T02:07:16.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.293+0000 7fc4c4fad640 1 -- 172.21.15.88:0/1157176217 wait complete. 2024-09-21T02:07:16.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.295+0000 7fc4c4fad640 1 Processor -- start 2024-09-21T02:07:16.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.295+0000 7fc4c4fad640 1 -- start start 2024-09-21T02:07:16.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.296+0000 7fc4c4fad640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4c0153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.296+0000 7fc4c4fad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c01eeed0 0x7fc4c00c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.296+0000 7fc4c4fad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c014ff50 0x7fc4c0150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.296+0000 7fc4c4fad640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0074ec0 con 0x7fc4a8005250 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.296+0000 7fc4c4fad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00c9310 con 0x7fc4c014ff50 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.296+0000 7fc4c4fad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c00754e0 con 0x7fc4c01eeed0 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4c0153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4c0153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39574/0 (socket says 172.21.15.88:39574) 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 -- 172.21.15.88:0/3156189440 learned_addr learned my addr 172.21.15.88:0/3156189440 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4bed9d640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c014ff50 0x7fc4c0150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4bdd9b640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c01eeed0 0x7fc4c00c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 -- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c01eeed0 msgr2=0x7fc4c00c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c01eeed0 0x7fc4c00c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 -- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c014ff50 msgr2=0x7fc4c0150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c014ff50 0x7fc4c0150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 -- 172.21.15.88:0/3156189440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01524f0 con 0x7fc4a8005250 2024-09-21T02:07:16.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4be59c640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4c0153d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0011820 tx=0x7fc4b0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:16.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4aeffd640 1 -- 172.21.15.88:0/3156189440 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b001f070 con 0x7fc4a8005250 2024-09-21T02:07:16.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4aeffd640 1 -- 172.21.15.88:0/3156189440 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b000fde0 con 0x7fc4a8005250 2024-09-21T02:07:16.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c00ccc80 con 0x7fc4a8005250 2024-09-21T02:07:16.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4aeffd640 1 -- 172.21.15.88:0/3156189440 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00126b0 con 0x7fc4a8005250 2024-09-21T02:07:16.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.297+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c0150e00 con 0x7fc4a8005250 2024-09-21T02:07:16.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.298+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c0158540 con 0x7fc4a8005250 2024-09-21T02:07:16.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.299+0000 7fc4aeffd640 1 -- 172.21.15.88:0/3156189440 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4b0022070 con 0x7fc4a8005250 2024-09-21T02:07:16.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.300+0000 7fc4aeffd640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc4a0078bc0 0x7fc4a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.300+0000 7fc4aeffd640 1 -- 172.21.15.88:0/3156189440 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc4b009b780 con 0x7fc4a8005250 2024-09-21T02:07:16.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.300+0000 7fc4bdd9b640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc4a0078bc0 0x7fc4a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.300+0000 7fc4bdd9b640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc4a0078bc0 0x7fc4a007b0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc4c01e0cf0 tx=0x7fc4b4006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:16.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.304+0000 7fc4aeffd640 1 -- 172.21.15.88:0/3156189440 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc4b0063500 con 0x7fc4a8005250 2024-09-21T02:07:16.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.477+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4c00ccf40 con 0x7fc4a0078bc0 2024-09-21T02:07:16.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.482+0000 7fc4aeffd640 1 -- 172.21.15.88:0/3156189440 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252956 (secure 0 0 0) 0x7fc4c00ccf40 con 0x7fc4a0078bc0 2024-09-21T02:07:16.484 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:16.485 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc4a0078bc0 msgr2=0x7fc4a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc4a0078bc0 0x7fc4a007b0d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc4c01e0cf0 tx=0x7fc4b4006b70 comp rx=0 tx=0).stop 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 msgr2=0x7fc4c0153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4c0153d10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0011820 tx=0x7fc4b0011ce0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 shutdown_connections 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc4a0078bc0 0x7fc4a007b0d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c014ff50 0x7fc4c0150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c01eeed0 0x7fc4c00c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 --2- 172.21.15.88:0/3156189440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8005250 0x7fc4c0153d10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 >> 172.21.15.88:0/3156189440 conn(0x7fc4c0070c30 msgr2=0x7fc4c0155380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 shutdown_connections 2024-09-21T02:07:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.485+0000 7fc4c4fad640 1 -- 172.21.15.88:0/3156189440 wait complete. 2024-09-21T02:07:16.497 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-21T02:07:14.874368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8213933,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":13134,"num_write_kb":25178465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14620,"ondisk_log_size":14620,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347944,"kb_used_data":18024,"kb_used_omap":12,"kb_used_meta":329779,"kb_avail":754626776,"statfs":{"total":773094113280,"available":772737818624,"internally_reserved":0,"allocated":18456576,"data_stored":17107362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":337694258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003656"},"pg_stats":[{"pgid":"3.3f","version":"23'159","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.474048+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:06:44.474048+0000","last_peered":"2024-09-21T02:06:44.474048+0000","last_clean":"2024-09-21T02:06:44.474048+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:06:44.474048+0000","last_undegraded":"2024-09-21T02:06:44.474048+0000","last_fullsized":"2024-09-21T02:06:44.474048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.266433+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:06:44.266433+0000","last_peered":"2024-09-21T02:06:44.266433+0000","last_clean":"2024-09-21T02:06:44.266433+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:06:44.266433+0000","last_undegraded":"2024-09-21T02:06:44.266433+0000","last_fullsized":"2024-09-21T02:06:44.266433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.550328+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:06:44.550328+0000","last_peered":"2024-09-21T02:06:44.550328+0000","last_clean":"2024-09-21T02:06:44.550328+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:06:44.550328+0000","last_undegraded":"2024-09-21T02:06:44.550328+0000","last_fullsized":"2024-09-21T02:06:44.550328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'185","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.561621+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:06:44.561621+0000","last_peered":"2024-09-21T02:06:44.561621+0000","last_clean":"2024-09-21T02:06:44.561621+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:06:44.561621+0000","last_undegraded":"2024-09-21T02:06:44.561621+0000","last_fullsized":"2024-09-21T02:06:44.561621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.479870+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:06:44.479870+0000","last_peered":"2024-09-21T02:06:44.479870+0000","last_clean":"2024-09-21T02:06:44.479870+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:06:44.479870+0000","last_undegraded":"2024-09-21T02:06:44.479870+0000","last_fullsized":"2024-09-21T02:06:44.479870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'156","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.530874+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:06:44.530874+0000","last_peered":"2024-09-21T02:06:44.530874+0000","last_clean":"2024-09-21T02:06:44.530874+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:06:44.530874+0000","last_undegraded":"2024-09-21T02:06:44.530874+0000","last_fullsized":"2024-09-21T02:06:44.530874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'183","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.223104+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:06:44.223104+0000","last_peered":"2024-09-21T02:06:44.223104+0000","last_clean":"2024-09-21T02:06:44.223104+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:06:44.223104+0000","last_undegraded":"2024-09-21T02:06:44.223104+0000","last_fullsized":"2024-09-21T02:06:44.223104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'195","reported_seq":213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.539730+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:06:44.539730+0000","last_peered":"2024-09-21T02:06:44.539730+0000","last_clean":"2024-09-21T02:06:44.539730+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:06:44.539730+0000","last_undegraded":"2024-09-21T02:06:44.539730+0000","last_fullsized":"2024-09-21T02:06:44.539730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'167","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.245878+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:06:44.245878+0000","last_peered":"2024-09-21T02:06:44.245878+0000","last_clean":"2024-09-21T02:06:44.245878+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:06:44.245878+0000","last_undegraded":"2024-09-21T02:06:44.245878+0000","last_fullsized":"2024-09-21T02:06:44.245878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.500726+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:06:44.500726+0000","last_peered":"2024-09-21T02:06:44.500726+0000","last_clean":"2024-09-21T02:06:44.500726+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:06:44.500726+0000","last_undegraded":"2024-09-21T02:06:44.500726+0000","last_fullsized":"2024-09-21T02:06:44.500726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'208","reported_seq":226,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.509048+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:06:44.509048+0000","last_peered":"2024-09-21T02:06:44.509048+0000","last_clean":"2024-09-21T02:06:44.509048+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:06:44.509048+0000","last_undegraded":"2024-09-21T02:06:44.509048+0000","last_fullsized":"2024-09-21T02:06:44.509048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'159","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.543706+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:06:44.543706+0000","last_peered":"2024-09-21T02:06:44.543706+0000","last_clean":"2024-09-21T02:06:44.543706+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:06:44.543706+0000","last_undegraded":"2024-09-21T02:06:44.543706+0000","last_fullsized":"2024-09-21T02:06:44.543706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'167","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.479687+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:06:44.479687+0000","last_peered":"2024-09-21T02:06:44.479687+0000","last_clean":"2024-09-21T02:06:44.479687+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:06:44.479687+0000","last_undegraded":"2024-09-21T02:06:44.479687+0000","last_fullsized":"2024-09-21T02:06:44.479687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.218722+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:06:44.218722+0000","last_peered":"2024-09-21T02:06:44.218722+0000","last_clean":"2024-09-21T02:06:44.218722+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:06:44.218722+0000","last_undegraded":"2024-09-21T02:06:44.218722+0000","last_fullsized":"2024-09-21T02:06:44.218722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490826+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:06:44.490826+0000","last_peered":"2024-09-21T02:06:44.490826+0000","last_clean":"2024-09-21T02:06:44.490826+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:06:44.490826+0000","last_undegraded":"2024-09-21T02:06:44.490826+0000","last_fullsized":"2024-09-21T02:06:44.490826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.258053+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:06:44.258053+0000","last_peered":"2024-09-21T02:06:44.258053+0000","last_clean":"2024-09-21T02:06:44.258053+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:06:44.258053+0000","last_undegraded":"2024-09-21T02:06:44.258053+0000","last_fullsized":"2024-09-21T02:06:44.258053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'168","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.554399+0000","last_change":"2024-09-21T02:04:45.212987+0000","last_active":"2024-09-21T02:06:44.554399+0000","last_peered":"2024-09-21T02:06:44.554399+0000","last_clean":"2024-09-21T02:06:44.554399+0000","last_became_active":"2024-09-21T02:04:45.212611+0000","last_became_peered":"2024-09-21T02:04:45.212611+0000","last_unstale":"2024-09-21T02:06:44.554399+0000","last_undegraded":"2024-09-21T02:06:44.554399+0000","last_fullsized":"2024-09-21T02:06:44.554399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:54.964291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'169","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490796+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:06:44.490796+0000","last_peered":"2024-09-21T02:06:44.490796+0000","last_clean":"2024-09-21T02:06:44.490796+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:06:44.490796+0000","last_undegraded":"2024-09-21T02:06:44.490796+0000","last_fullsized":"2024-09-21T02:06:44.490796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'198","reported_seq":216,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.513284+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:06:44.513284+0000","last_peered":"2024-09-21T02:06:44.513284+0000","last_clean":"2024-09-21T02:06:44.513284+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:06:44.513284+0000","last_undegraded":"2024-09-21T02:06:44.513284+0000","last_fullsized":"2024-09-21T02:06:44.513284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.548720+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:06:44.548720+0000","last_peered":"2024-09-21T02:06:44.548720+0000","last_clean":"2024-09-21T02:06:44.548720+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:06:44.548720+0000","last_undegraded":"2024-09-21T02:06:44.548720+0000","last_fullsized":"2024-09-21T02:06:44.548720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'143","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.258179+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:06:44.258179+0000","last_peered":"2024-09-21T02:06:44.258179+0000","last_clean":"2024-09-21T02:06:44.258179+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:06:44.258179+0000","last_undegraded":"2024-09-21T02:06:44.258179+0000","last_fullsized":"2024-09-21T02:06:44.258179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'189","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.493538+0000","last_change":"2024-09-21T02:04:45.212961+0000","last_active":"2024-09-21T02:06:44.493538+0000","last_peered":"2024-09-21T02:06:44.493538+0000","last_clean":"2024-09-21T02:06:44.493538+0000","last_became_active":"2024-09-21T02:04:45.212587+0000","last_became_peered":"2024-09-21T02:04:45.212587+0000","last_unstale":"2024-09-21T02:06:44.493538+0000","last_undegraded":"2024-09-21T02:06:44.493538+0000","last_fullsized":"2024-09-21T02:06:44.493538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:12.690374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"23'142","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.253560+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:06:44.253560+0000","last_peered":"2024-09-21T02:06:44.253560+0000","last_clean":"2024-09-21T02:06:44.253560+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:06:44.253560+0000","last_undegraded":"2024-09-21T02:06:44.253560+0000","last_fullsized":"2024-09-21T02:06:44.253560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'201","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.533311+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:06:44.533311+0000","last_peered":"2024-09-21T02:06:44.533311+0000","last_clean":"2024-09-21T02:06:44.533311+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:06:44.533311+0000","last_undegraded":"2024-09-21T02:06:44.533311+0000","last_fullsized":"2024-09-21T02:06:44.533311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.262090+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:06:44.262090+0000","last_peered":"2024-09-21T02:06:44.262090+0000","last_clean":"2024-09-21T02:06:44.262090+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:06:44.262090+0000","last_undegraded":"2024-09-21T02:06:44.262090+0000","last_fullsized":"2024-09-21T02:06:44.262090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'202","reported_seq":220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.221781+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:06:44.221781+0000","last_peered":"2024-09-21T02:06:44.221781+0000","last_clean":"2024-09-21T02:06:44.221781+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:06:44.221781+0000","last_undegraded":"2024-09-21T02:06:44.221781+0000","last_fullsized":"2024-09-21T02:06:44.221781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.273171+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:06:44.273171+0000","last_peered":"2024-09-21T02:06:44.273171+0000","last_clean":"2024-09-21T02:06:44.273171+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:06:44.273171+0000","last_undegraded":"2024-09-21T02:06:44.273171+0000","last_fullsized":"2024-09-21T02:06:44.273171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.236933+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:06:44.236933+0000","last_peered":"2024-09-21T02:06:44.236933+0000","last_clean":"2024-09-21T02:06:44.236933+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:06:44.236933+0000","last_undegraded":"2024-09-21T02:06:44.236933+0000","last_fullsized":"2024-09-21T02:06:44.236933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'127","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.232151+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:06:44.232151+0000","last_peered":"2024-09-21T02:06:44.232151+0000","last_clean":"2024-09-21T02:06:44.232151+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:06:44.232151+0000","last_undegraded":"2024-09-21T02:06:44.232151+0000","last_fullsized":"2024-09-21T02:06:44.232151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'145","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.228888+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:06:44.228888+0000","last_peered":"2024-09-21T02:06:44.228888+0000","last_clean":"2024-09-21T02:06:44.228888+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:06:44.228888+0000","last_undegraded":"2024-09-21T02:06:44.228888+0000","last_fullsized":"2024-09-21T02:06:44.228888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'146","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.226313+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:06:44.226313+0000","last_peered":"2024-09-21T02:06:44.226313+0000","last_clean":"2024-09-21T02:06:44.226313+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:06:44.226313+0000","last_undegraded":"2024-09-21T02:06:44.226313+0000","last_fullsized":"2024-09-21T02:06:44.226313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.506925+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:06:44.506925+0000","last_peered":"2024-09-21T02:06:44.506925+0000","last_clean":"2024-09-21T02:06:44.506925+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:06:44.506925+0000","last_undegraded":"2024-09-21T02:06:44.506925+0000","last_fullsized":"2024-09-21T02:06:44.506925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'142","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.255204+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:06:44.255204+0000","last_peered":"2024-09-21T02:06:44.255204+0000","last_clean":"2024-09-21T02:06:44.255204+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:06:44.255204+0000","last_undegraded":"2024-09-21T02:06:44.255204+0000","last_fullsized":"2024-09-21T02:06:44.255204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1643","reported_seq":1663,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:48.101923+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:06:48.101923+0000","last_peered":"2024-09-21T02:06:48.101923+0000","last_clean":"2024-09-21T02:06:48.101923+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:06:48.101923+0000","last_undegraded":"2024-09-21T02:06:48.101923+0000","last_fullsized":"2024-09-21T02:06:48.101923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1643,"log_dups_size":0,"ondisk_log_size":1643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3424840,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1640,"num_write_kb":4925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.218892+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:06:44.218892+0000","last_peered":"2024-09-21T02:06:44.218892+0000","last_clean":"2024-09-21T02:06:44.218892+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:06:44.218892+0000","last_undegraded":"2024-09-21T02:06:44.218892+0000","last_fullsized":"2024-09-21T02:06:44.218892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.279847+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:04:45.279847+0000","last_peered":"2024-09-21T02:04:45.279847+0000","last_clean":"2024-09-21T02:04:45.279847+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:04:45.279847+0000","last_undegraded":"2024-09-21T02:04:45.279847+0000","last_fullsized":"2024-09-21T02:04:45.279847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280546+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:04:45.280546+0000","last_peered":"2024-09-21T02:04:45.280546+0000","last_clean":"2024-09-21T02:04:45.280546+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:04:45.280546+0000","last_undegraded":"2024-09-21T02:04:45.280546+0000","last_fullsized":"2024-09-21T02:04:45.280546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'182","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.493437+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:06:44.493437+0000","last_peered":"2024-09-21T02:06:44.493437+0000","last_clean":"2024-09-21T02:06:44.493437+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:06:44.493437+0000","last_undegraded":"2024-09-21T02:06:44.493437+0000","last_fullsized":"2024-09-21T02:06:44.493437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205335+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:04:45.205335+0000","last_peered":"2024-09-21T02:04:45.205335+0000","last_clean":"2024-09-21T02:04:45.205335+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:04:45.205335+0000","last_undegraded":"2024-09-21T02:04:45.205335+0000","last_fullsized":"2024-09-21T02:04:45.205335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.504661+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:06:44.504661+0000","last_peered":"2024-09-21T02:06:44.504661+0000","last_clean":"2024-09-21T02:06:44.504661+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:06:44.504661+0000","last_undegraded":"2024-09-21T02:06:44.504661+0000","last_fullsized":"2024-09-21T02:06:44.504661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.142553+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:05:47.142553+0000","last_peered":"2024-09-21T02:05:47.142553+0000","last_clean":"2024-09-21T02:05:47.142553+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:05:47.142553+0000","last_undegraded":"2024-09-21T02:05:47.142553+0000","last_fullsized":"2024-09-21T02:05:47.142553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'241","reported_seq":253,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.541162+0000","last_change":"2024-09-21T02:04:45.212312+0000","last_active":"2024-09-21T02:06:44.541162+0000","last_peered":"2024-09-21T02:06:44.541162+0000","last_clean":"2024-09-21T02:06:44.541162+0000","last_became_active":"2024-09-21T02:04:45.211981+0000","last_became_peered":"2024-09-21T02:04:45.211981+0000","last_unstale":"2024-09-21T02:06:44.541162+0000","last_undegraded":"2024-09-21T02:06:44.541162+0000","last_fullsized":"2024-09-21T02:06:44.541162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.148945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205736+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:04:45.205736+0000","last_peered":"2024-09-21T02:04:45.205736+0000","last_clean":"2024-09-21T02:04:45.205736+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:04:45.205736+0000","last_undegraded":"2024-09-21T02:04:45.205736+0000","last_fullsized":"2024-09-21T02:04:45.205736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'151","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.264984+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:06:44.264984+0000","last_peered":"2024-09-21T02:06:44.264984+0000","last_clean":"2024-09-21T02:06:44.264984+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:06:44.264984+0000","last_undegraded":"2024-09-21T02:06:44.264984+0000","last_fullsized":"2024-09-21T02:06:44.264984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:45.329388+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:06:45.329388+0000","last_peered":"2024-09-21T02:06:45.329388+0000","last_clean":"2024-09-21T02:06:45.329388+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:06:45.329388+0000","last_undegraded":"2024-09-21T02:06:45.329388+0000","last_fullsized":"2024-09-21T02:06:45.329388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.231890+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:06:44.231890+0000","last_peered":"2024-09-21T02:06:44.231890+0000","last_clean":"2024-09-21T02:06:44.231890+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:06:44.231890+0000","last_undegraded":"2024-09-21T02:06:44.231890+0000","last_fullsized":"2024-09-21T02:06:44.231890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'2017","reported_seq":2037,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:19.585213+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:06:19.585213+0000","last_peered":"2024-09-21T02:06:19.585213+0000","last_clean":"2024-09-21T02:06:19.585213+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:06:19.585213+0000","last_undegraded":"2024-09-21T02:06:19.585213+0000","last_fullsized":"2024-09-21T02:06:19.585213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2017,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'170","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.488411+0000","last_change":"2024-09-21T02:04:45.213511+0000","last_active":"2024-09-21T02:06:44.488411+0000","last_peered":"2024-09-21T02:06:44.488411+0000","last_clean":"2024-09-21T02:06:44.488411+0000","last_became_active":"2024-09-21T02:04:45.213182+0000","last_became_peered":"2024-09-21T02:04:45.213182+0000","last_unstale":"2024-09-21T02:06:44.488411+0000","last_undegraded":"2024-09-21T02:06:44.488411+0000","last_fullsized":"2024-09-21T02:06:44.488411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:11.613618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280485+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:45.280485+0000","last_peered":"2024-09-21T02:04:45.280485+0000","last_clean":"2024-09-21T02:04:45.280485+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:45.280485+0000","last_undegraded":"2024-09-21T02:04:45.280485+0000","last_fullsized":"2024-09-21T02:04:45.280485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.221865+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:06:44.221865+0000","last_peered":"2024-09-21T02:06:44.221865+0000","last_clean":"2024-09-21T02:06:44.221865+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:06:44.221865+0000","last_undegraded":"2024-09-21T02:06:44.221865+0000","last_fullsized":"2024-09-21T02:06:44.221865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205637+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:04:45.205637+0000","last_peered":"2024-09-21T02:04:45.205637+0000","last_clean":"2024-09-21T02:04:45.205637+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:04:45.205637+0000","last_undegraded":"2024-09-21T02:04:45.205637+0000","last_fullsized":"2024-09-21T02:04:45.205637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'186","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.503215+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:06:44.503215+0000","last_peered":"2024-09-21T02:06:44.503215+0000","last_clean":"2024-09-21T02:06:44.503215+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:06:44.503215+0000","last_undegraded":"2024-09-21T02:06:44.503215+0000","last_fullsized":"2024-09-21T02:06:44.503215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.279800+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:04:45.279800+0000","last_peered":"2024-09-21T02:04:45.279800+0000","last_clean":"2024-09-21T02:04:45.279800+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:04:45.279800+0000","last_undegraded":"2024-09-21T02:04:45.279800+0000","last_fullsized":"2024-09-21T02:04:45.279800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'191","reported_seq":209,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.517881+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:06:44.517881+0000","last_peered":"2024-09-21T02:06:44.517881+0000","last_clean":"2024-09-21T02:06:44.517881+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:06:44.517881+0000","last_undegraded":"2024-09-21T02:06:44.517881+0000","last_fullsized":"2024-09-21T02:06:44.517881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'16","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:16.689517+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:06:16.689517+0000","last_peered":"2024-09-21T02:06:16.689517+0000","last_clean":"2024-09-21T02:06:16.689517+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:06:16.689517+0000","last_undegraded":"2024-09-21T02:06:16.689517+0000","last_fullsized":"2024-09-21T02:06:16.689517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'209","reported_seq":227,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.230309+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:06:44.230309+0000","last_peered":"2024-09-21T02:06:44.230309+0000","last_clean":"2024-09-21T02:06:44.230309+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:06:44.230309+0000","last_undegraded":"2024-09-21T02:06:44.230309+0000","last_fullsized":"2024-09-21T02:06:44.230309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.203910+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:04:45.203910+0000","last_peered":"2024-09-21T02:04:45.203910+0000","last_clean":"2024-09-21T02:04:45.203910+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:04:45.203910+0000","last_undegraded":"2024-09-21T02:04:45.203910+0000","last_fullsized":"2024-09-21T02:04:45.203910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.566471+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:06:44.566471+0000","last_peered":"2024-09-21T02:06:44.566471+0000","last_clean":"2024-09-21T02:06:44.566471+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:06:44.566471+0000","last_undegraded":"2024-09-21T02:06:44.566471+0000","last_fullsized":"2024-09-21T02:06:44.566471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208099+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:45.208099+0000","last_peered":"2024-09-21T02:04:45.208099+0000","last_clean":"2024-09-21T02:04:45.208099+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:45.208099+0000","last_undegraded":"2024-09-21T02:04:45.208099+0000","last_fullsized":"2024-09-21T02:04:45.208099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.212614+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:06:44.212614+0000","last_peered":"2024-09-21T02:06:44.212614+0000","last_clean":"2024-09-21T02:06:44.212614+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:06:44.212614+0000","last_undegraded":"2024-09-21T02:06:44.212614+0000","last_fullsized":"2024-09-21T02:06:44.212614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208098+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:04:45.208098+0000","last_peered":"2024-09-21T02:04:45.208098+0000","last_clean":"2024-09-21T02:04:45.208098+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:04:45.208098+0000","last_undegraded":"2024-09-21T02:04:45.208098+0000","last_fullsized":"2024-09-21T02:04:45.208098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'159","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.513761+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:06:44.513761+0000","last_peered":"2024-09-21T02:06:44.513761+0000","last_clean":"2024-09-21T02:06:44.513761+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:06:44.513761+0000","last_undegraded":"2024-09-21T02:06:44.513761+0000","last_fullsized":"2024-09-21T02:06:44.513761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207953+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:04:45.207953+0000","last_peered":"2024-09-21T02:04:45.207953+0000","last_clean":"2024-09-21T02:04:45.207953+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:04:45.207953+0000","last_undegraded":"2024-09-21T02:04:45.207953+0000","last_fullsized":"2024-09-21T02:04:45.207953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.270827+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:06:44.270827+0000","last_peered":"2024-09-21T02:06:44.270827+0000","last_clean":"2024-09-21T02:06:44.270827+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:06:44.270827+0000","last_undegraded":"2024-09-21T02:06:44.270827+0000","last_fullsized":"2024-09-21T02:06:44.270827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211129+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:04:45.211129+0000","last_peered":"2024-09-21T02:04:45.211129+0000","last_clean":"2024-09-21T02:04:45.211129+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:04:45.211129+0000","last_undegraded":"2024-09-21T02:04:45.211129+0000","last_fullsized":"2024-09-21T02:04:45.211129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205332+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:04:45.205332+0000","last_peered":"2024-09-21T02:04:45.205332+0000","last_clean":"2024-09-21T02:04:45.205332+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:04:45.205332+0000","last_undegraded":"2024-09-21T02:04:45.205332+0000","last_fullsized":"2024-09-21T02:04:45.205332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.211254+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:06:44.211254+0000","last_peered":"2024-09-21T02:06:44.211254+0000","last_clean":"2024-09-21T02:06:44.211254+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:06:44.211254+0000","last_undegraded":"2024-09-21T02:06:44.211254+0000","last_fullsized":"2024-09-21T02:06:44.211254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210923+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:04:45.210923+0000","last_peered":"2024-09-21T02:04:45.210923+0000","last_clean":"2024-09-21T02:04:45.210923+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:04:45.210923+0000","last_undegraded":"2024-09-21T02:04:45.210923+0000","last_fullsized":"2024-09-21T02:04:45.210923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.562728+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:06:44.562728+0000","last_peered":"2024-09-21T02:06:44.562728+0000","last_clean":"2024-09-21T02:06:44.562728+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:06:44.562728+0000","last_undegraded":"2024-09-21T02:06:44.562728+0000","last_fullsized":"2024-09-21T02:06:44.562728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:05.466556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'10","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:47.179158+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:06:47.179158+0000","last_peered":"2024-09-21T02:06:47.179158+0000","last_clean":"2024-09-21T02:06:47.179158+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:06:47.179158+0000","last_undegraded":"2024-09-21T02:06:47.179158+0000","last_fullsized":"2024-09-21T02:06:47.179158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":595,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.494702+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:06:44.494702+0000","last_peered":"2024-09-21T02:06:44.494702+0000","last_clean":"2024-09-21T02:06:44.494702+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:06:44.494702+0000","last_undegraded":"2024-09-21T02:06:44.494702+0000","last_fullsized":"2024-09-21T02:06:44.494702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205672+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:04:45.205672+0000","last_peered":"2024-09-21T02:04:45.205672+0000","last_clean":"2024-09-21T02:04:45.205672+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:04:45.205672+0000","last_undegraded":"2024-09-21T02:04:45.205672+0000","last_fullsized":"2024-09-21T02:04:45.205672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.555314+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:06:44.555314+0000","last_peered":"2024-09-21T02:06:44.555314+0000","last_clean":"2024-09-21T02:06:44.555314+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:06:44.555314+0000","last_undegraded":"2024-09-21T02:06:44.555314+0000","last_fullsized":"2024-09-21T02:06:44.555314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207553+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:04:45.207553+0000","last_peered":"2024-09-21T02:04:45.207553+0000","last_clean":"2024-09-21T02:04:45.207553+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:04:45.207553+0000","last_undegraded":"2024-09-21T02:04:45.207553+0000","last_fullsized":"2024-09-21T02:04:45.207553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'155","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.258318+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:06:44.258318+0000","last_peered":"2024-09-21T02:06:44.258318+0000","last_clean":"2024-09-21T02:06:44.258318+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:06:44.258318+0000","last_undegraded":"2024-09-21T02:06:44.258318+0000","last_fullsized":"2024-09-21T02:06:44.258318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211858+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:04:45.211858+0000","last_peered":"2024-09-21T02:04:45.211858+0000","last_clean":"2024-09-21T02:04:45.211858+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:04:45.211858+0000","last_undegraded":"2024-09-21T02:04:45.211858+0000","last_fullsized":"2024-09-21T02:04:45.211858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.555740+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:06:44.555740+0000","last_peered":"2024-09-21T02:06:44.555740+0000","last_clean":"2024-09-21T02:06:44.555740+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:06:44.555740+0000","last_undegraded":"2024-09-21T02:06:44.555740+0000","last_fullsized":"2024-09-21T02:06:44.555740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'21","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:43.130438+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:06:43.130438+0000","last_peered":"2024-09-21T02:06:43.130438+0000","last_clean":"2024-09-21T02:06:43.130438+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:06:43.130438+0000","last_undegraded":"2024-09-21T02:06:43.130438+0000","last_fullsized":"2024-09-21T02:06:43.130438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'161","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.495833+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:06:44.495833+0000","last_peered":"2024-09-21T02:06:44.495833+0000","last_clean":"2024-09-21T02:06:44.495833+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:06:44.495833+0000","last_undegraded":"2024-09-21T02:06:44.495833+0000","last_fullsized":"2024-09-21T02:06:44.495833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:09.584145+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:06:09.584145+0000","last_peered":"2024-09-21T02:06:09.584145+0000","last_clean":"2024-09-21T02:06:09.584145+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:06:09.584145+0000","last_undegraded":"2024-09-21T02:06:09.584145+0000","last_fullsized":"2024-09-21T02:06:09.584145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.520907+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:06:44.520907+0000","last_peered":"2024-09-21T02:06:44.520907+0000","last_clean":"2024-09-21T02:06:44.520907+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:06:44.520907+0000","last_undegraded":"2024-09-21T02:06:44.520907+0000","last_fullsized":"2024-09-21T02:06:44.520907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210942+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:04:45.210942+0000","last_peered":"2024-09-21T02:04:45.210942+0000","last_clean":"2024-09-21T02:04:45.210942+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:04:45.210942+0000","last_undegraded":"2024-09-21T02:04:45.210942+0000","last_fullsized":"2024-09-21T02:04:45.210942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.558277+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:06:44.558277+0000","last_peered":"2024-09-21T02:06:44.558277+0000","last_clean":"2024-09-21T02:06:44.558277+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:06:44.558277+0000","last_undegraded":"2024-09-21T02:06:44.558277+0000","last_fullsized":"2024-09-21T02:06:44.558277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207751+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:04:45.207751+0000","last_peered":"2024-09-21T02:04:45.207751+0000","last_clean":"2024-09-21T02:04:45.207751+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:04:45.207751+0000","last_undegraded":"2024-09-21T02:04:45.207751+0000","last_fullsized":"2024-09-21T02:04:45.207751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.214359+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:06:44.214359+0000","last_peered":"2024-09-21T02:06:44.214359+0000","last_clean":"2024-09-21T02:06:44.214359+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:06:44.214359+0000","last_undegraded":"2024-09-21T02:06:44.214359+0000","last_fullsized":"2024-09-21T02:06:44.214359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207926+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:04:45.207926+0000","last_peered":"2024-09-21T02:04:45.207926+0000","last_clean":"2024-09-21T02:04:45.207926+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:04:45.207926+0000","last_undegraded":"2024-09-21T02:04:45.207926+0000","last_fullsized":"2024-09-21T02:04:45.207926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'156","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.499249+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:06:44.499249+0000","last_peered":"2024-09-21T02:06:44.499249+0000","last_clean":"2024-09-21T02:06:44.499249+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:06:44.499249+0000","last_undegraded":"2024-09-21T02:06:44.499249+0000","last_fullsized":"2024-09-21T02:06:44.499249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.141927+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:05:47.141927+0000","last_peered":"2024-09-21T02:05:47.141927+0000","last_clean":"2024-09-21T02:05:47.141927+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:05:47.141927+0000","last_undegraded":"2024-09-21T02:05:47.141927+0000","last_fullsized":"2024-09-21T02:05:47.141927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.537214+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:06:44.537214+0000","last_peered":"2024-09-21T02:06:44.537214+0000","last_clean":"2024-09-21T02:06:44.537214+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:06:44.537214+0000","last_undegraded":"2024-09-21T02:06:44.537214+0000","last_fullsized":"2024-09-21T02:06:44.537214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'190","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.515015+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:06:44.515015+0000","last_peered":"2024-09-21T02:06:44.515015+0000","last_clean":"2024-09-21T02:06:44.515015+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:06:44.515015+0000","last_undegraded":"2024-09-21T02:06:44.515015+0000","last_fullsized":"2024-09-21T02:06:44.515015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211388+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:45.211388+0000","last_peered":"2024-09-21T02:04:45.211388+0000","last_clean":"2024-09-21T02:04:45.211388+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:45.211388+0000","last_undegraded":"2024-09-21T02:04:45.211388+0000","last_fullsized":"2024-09-21T02:04:45.211388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.487942+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:06:44.487942+0000","last_peered":"2024-09-21T02:06:44.487942+0000","last_clean":"2024-09-21T02:06:44.487942+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:06:44.487942+0000","last_undegraded":"2024-09-21T02:06:44.487942+0000","last_fullsized":"2024-09-21T02:06:44.487942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:08.902449+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:06:08.902449+0000","last_peered":"2024-09-21T02:06:08.902449+0000","last_clean":"2024-09-21T02:06:08.902449+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:06:08.902449+0000","last_undegraded":"2024-09-21T02:06:08.902449+0000","last_fullsized":"2024-09-21T02:06:08.902449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.244570+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:06:44.244570+0000","last_peered":"2024-09-21T02:06:44.244570+0000","last_clean":"2024-09-21T02:06:44.244570+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:06:44.244570+0000","last_undegraded":"2024-09-21T02:06:44.244570+0000","last_fullsized":"2024-09-21T02:06:44.244570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280872+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:04:45.280872+0000","last_peered":"2024-09-21T02:04:45.280872+0000","last_clean":"2024-09-21T02:04:45.280872+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:04:45.280872+0000","last_undegraded":"2024-09-21T02:04:45.280872+0000","last_fullsized":"2024-09-21T02:04:45.280872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490694+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:06:44.490694+0000","last_peered":"2024-09-21T02:06:44.490694+0000","last_clean":"2024-09-21T02:06:44.490694+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:06:44.490694+0000","last_undegraded":"2024-09-21T02:06:44.490694+0000","last_fullsized":"2024-09-21T02:06:44.490694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280763+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:04:45.280763+0000","last_peered":"2024-09-21T02:04:45.280763+0000","last_clean":"2024-09-21T02:04:45.280763+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:04:45.280763+0000","last_undegraded":"2024-09-21T02:04:45.280763+0000","last_fullsized":"2024-09-21T02:04:45.280763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7623565,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3817,"num_write_kb":11013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15400960,"data_stored":15247130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3793,"ondisk_log_size":3793,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57564,"kb_used_data":4172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314276,"statfs":{"total":96636764160,"available":96577818624,"internally_reserved":0,"allocated":4272128,"data_stored":4100839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":86315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":85907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83228,"kb_used_data":7692,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94288612,"statfs":{"total":96636764160,"available":96551538688,"internally_reserved":0,"allocated":7876608,"data_stored":7705348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":86260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":85881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53572,"kb_used_data":4340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318268,"statfs":{"total":96636764160,"available":96581906432,"internally_reserved":0,"allocated":4444160,"data_stored":4280847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":843776,"data_stored":675965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7643136,"data_stored":7620411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3444736,"data_stored":3425534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:07:16.501 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:07:16.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.649+0000 7f9bf7539640 1 -- 172.21.15.88:0/3647568988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 msgr2=0x7f9bf00c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.649+0000 7f9bf7539640 1 --2- 172.21.15.88:0/3647568988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 0x7f9bf00c41c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9be0005fd0 tx=0x7f9be002a670 comp rx=0 tx=0).stop 2024-09-21T02:07:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.649+0000 7f9bf7539640 1 -- 172.21.15.88:0/3647568988 shutdown_connections 2024-09-21T02:07:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.649+0000 7f9bf7539640 1 --2- 172.21.15.88:0/3647568988 >> v2:172.21.15.195:3300/0 conn(0x7f9bf0074490 0x7f9bf00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.649+0000 7f9bf7539640 1 --2- 172.21.15.88:0/3647568988 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 0x7f9bf00c41c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.649+0000 7f9bf7539640 1 -- 172.21.15.88:0/3647568988 >> 172.21.15.88:0/3647568988 conn(0x7f9bf00ca8d0 msgr2=0x7f9bf00cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.650+0000 7f9bf7539640 1 -- 172.21.15.88:0/3647568988 shutdown_connections 2024-09-21T02:07:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.650+0000 7f9bf7539640 1 -- 172.21.15.88:0/3647568988 wait complete. 2024-09-21T02:07:16.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.651+0000 7f9bf7539640 1 Processor -- start 2024-09-21T02:07:16.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.651+0000 7f9bf7539640 1 -- start start 2024-09-21T02:07:16.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.652+0000 7f9bf7539640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf0074490 0x7f9bf014a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.652+0000 7f9bf7539640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 0x7f9bf0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.652+0000 7f9bf7539640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bf01526a0 0x7f9bf014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.652+0000 7f9bf7539640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00cb770 con 0x7f9bf0074490 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.652+0000 7f9bf7539640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf0158a40 con 0x7f9bf01526a0 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.652+0000 7f9bf7539640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf00759e0 con 0x7f9bf00c97e0 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf0074490 0x7f9bf014a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf0074490 0x7f9bf014a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39576/0 (socket says 172.21.15.88:39576) 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 -- 172.21.15.88:0/1740446168 learned_addr learned my addr 172.21.15.88:0/1740446168 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf5ad6640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bf01526a0 0x7f9bf014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf4ad4640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 0x7f9bf0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 -- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 msgr2=0x7f9bf0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 0x7f9bf0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 -- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bf01526a0 msgr2=0x7f9bf014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bf01526a0 0x7f9bf014ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 -- 172.21.15.88:0/1740446168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf015b6e0 con 0x7f9bf0074490 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf4ad4640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 0x7f9bf0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf52d5640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf0074490 0x7f9bf014a300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eab0 tx=0x7f9bec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9be5ffb640 1 -- 172.21.15.88:0/1740446168 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec001f70 con 0x7f9bf0074490 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9be5ffb640 1 -- 172.21.15.88:0/1740446168 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bec003970 con 0x7f9bf0074490 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf01ef300 con 0x7f9bf0074490 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.653+0000 7f9be5ffb640 1 -- 172.21.15.88:0/1740446168 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec003da0 con 0x7f9bf0074490 2024-09-21T02:07:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.654+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf01ef8c0 con 0x7f9bf0074490 2024-09-21T02:07:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.654+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf01ef0f0 con 0x7f9bf0074490 2024-09-21T02:07:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.655+0000 7f9be5ffb640 1 -- 172.21.15.88:0/1740446168 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9bec0050d0 con 0x7f9bf0074490 2024-09-21T02:07:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.656+0000 7f9be5ffb640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9bd4078bc0 0x7f9bd407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.656+0000 7f9be5ffb640 1 -- 172.21.15.88:0/1740446168 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9bec09d3f0 con 0x7f9bf0074490 2024-09-21T02:07:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.656+0000 7f9bf4ad4640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9bd4078bc0 0x7f9bd407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.656+0000 7f9bf4ad4640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9bd4078bc0 0x7f9bd407b0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9be0033000 tx=0x7f9be0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.660+0000 7f9be5ffb640 1 -- 172.21.15.88:0/1740446168 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9bec016020 con 0x7f9bf0074490 2024-09-21T02:07:16.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.836+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bf015a2a0 con 0x7f9bd4078bc0 2024-09-21T02:07:16.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.841+0000 7f9be5ffb640 1 -- 172.21.15.88:0/1740446168 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252956 (secure 0 0 0) 0x7f9bf015a2a0 con 0x7f9bd4078bc0 2024-09-21T02:07:16.843 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:16.844 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:07:16.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.843+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9bd4078bc0 msgr2=0x7f9bd407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.843+0000 7f9bf7539640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9bd4078bc0 0x7f9bd407b0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9be0033000 tx=0x7f9be0004730 comp rx=0 tx=0).stop 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.843+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf0074490 msgr2=0x7f9bf014a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.843+0000 7f9bf7539640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf0074490 0x7f9bf014a300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eab0 tx=0x7f9bec00eeb0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 shutdown_connections 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9bd4078bc0 0x7f9bd407b0d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9bf01526a0 0x7f9bf014ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9bf00c97e0 0x7f9bf0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 --2- 172.21.15.88:0/1740446168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf0074490 0x7f9bf014a300 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 >> 172.21.15.88:0/1740446168 conn(0x7f9bf00ca8d0 msgr2=0x7f9bf0147a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 shutdown_connections 2024-09-21T02:07:16.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:16.844+0000 7f9bf7539640 1 -- 172.21.15.88:0/1740446168 wait complete. 2024-09-21T02:07:16.855 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-21T02:07:14.874368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8213933,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":13134,"num_write_kb":25178465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14620,"ondisk_log_size":14620,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347944,"kb_used_data":18024,"kb_used_omap":12,"kb_used_meta":329779,"kb_avail":754626776,"statfs":{"total":773094113280,"available":772737818624,"internally_reserved":0,"allocated":18456576,"data_stored":17107362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":337694258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003656"},"pg_stats":[{"pgid":"3.3f","version":"23'159","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.474048+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:06:44.474048+0000","last_peered":"2024-09-21T02:06:44.474048+0000","last_clean":"2024-09-21T02:06:44.474048+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:06:44.474048+0000","last_undegraded":"2024-09-21T02:06:44.474048+0000","last_fullsized":"2024-09-21T02:06:44.474048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.266433+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:06:44.266433+0000","last_peered":"2024-09-21T02:06:44.266433+0000","last_clean":"2024-09-21T02:06:44.266433+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:06:44.266433+0000","last_undegraded":"2024-09-21T02:06:44.266433+0000","last_fullsized":"2024-09-21T02:06:44.266433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.550328+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:06:44.550328+0000","last_peered":"2024-09-21T02:06:44.550328+0000","last_clean":"2024-09-21T02:06:44.550328+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:06:44.550328+0000","last_undegraded":"2024-09-21T02:06:44.550328+0000","last_fullsized":"2024-09-21T02:06:44.550328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'185","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.561621+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:06:44.561621+0000","last_peered":"2024-09-21T02:06:44.561621+0000","last_clean":"2024-09-21T02:06:44.561621+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:06:44.561621+0000","last_undegraded":"2024-09-21T02:06:44.561621+0000","last_fullsized":"2024-09-21T02:06:44.561621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.479870+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:06:44.479870+0000","last_peered":"2024-09-21T02:06:44.479870+0000","last_clean":"2024-09-21T02:06:44.479870+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:06:44.479870+0000","last_undegraded":"2024-09-21T02:06:44.479870+0000","last_fullsized":"2024-09-21T02:06:44.479870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'156","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.530874+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:06:44.530874+0000","last_peered":"2024-09-21T02:06:44.530874+0000","last_clean":"2024-09-21T02:06:44.530874+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:06:44.530874+0000","last_undegraded":"2024-09-21T02:06:44.530874+0000","last_fullsized":"2024-09-21T02:06:44.530874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'183","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.223104+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:06:44.223104+0000","last_peered":"2024-09-21T02:06:44.223104+0000","last_clean":"2024-09-21T02:06:44.223104+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:06:44.223104+0000","last_undegraded":"2024-09-21T02:06:44.223104+0000","last_fullsized":"2024-09-21T02:06:44.223104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'195","reported_seq":213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.539730+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:06:44.539730+0000","last_peered":"2024-09-21T02:06:44.539730+0000","last_clean":"2024-09-21T02:06:44.539730+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:06:44.539730+0000","last_undegraded":"2024-09-21T02:06:44.539730+0000","last_fullsized":"2024-09-21T02:06:44.539730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'167","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.245878+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:06:44.245878+0000","last_peered":"2024-09-21T02:06:44.245878+0000","last_clean":"2024-09-21T02:06:44.245878+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:06:44.245878+0000","last_undegraded":"2024-09-21T02:06:44.245878+0000","last_fullsized":"2024-09-21T02:06:44.245878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.500726+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:06:44.500726+0000","last_peered":"2024-09-21T02:06:44.500726+0000","last_clean":"2024-09-21T02:06:44.500726+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:06:44.500726+0000","last_undegraded":"2024-09-21T02:06:44.500726+0000","last_fullsized":"2024-09-21T02:06:44.500726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'208","reported_seq":226,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.509048+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:06:44.509048+0000","last_peered":"2024-09-21T02:06:44.509048+0000","last_clean":"2024-09-21T02:06:44.509048+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:06:44.509048+0000","last_undegraded":"2024-09-21T02:06:44.509048+0000","last_fullsized":"2024-09-21T02:06:44.509048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'159","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.543706+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:06:44.543706+0000","last_peered":"2024-09-21T02:06:44.543706+0000","last_clean":"2024-09-21T02:06:44.543706+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:06:44.543706+0000","last_undegraded":"2024-09-21T02:06:44.543706+0000","last_fullsized":"2024-09-21T02:06:44.543706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'167","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.479687+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:06:44.479687+0000","last_peered":"2024-09-21T02:06:44.479687+0000","last_clean":"2024-09-21T02:06:44.479687+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:06:44.479687+0000","last_undegraded":"2024-09-21T02:06:44.479687+0000","last_fullsized":"2024-09-21T02:06:44.479687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.218722+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:06:44.218722+0000","last_peered":"2024-09-21T02:06:44.218722+0000","last_clean":"2024-09-21T02:06:44.218722+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:06:44.218722+0000","last_undegraded":"2024-09-21T02:06:44.218722+0000","last_fullsized":"2024-09-21T02:06:44.218722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490826+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:06:44.490826+0000","last_peered":"2024-09-21T02:06:44.490826+0000","last_clean":"2024-09-21T02:06:44.490826+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:06:44.490826+0000","last_undegraded":"2024-09-21T02:06:44.490826+0000","last_fullsized":"2024-09-21T02:06:44.490826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.258053+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:06:44.258053+0000","last_peered":"2024-09-21T02:06:44.258053+0000","last_clean":"2024-09-21T02:06:44.258053+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:06:44.258053+0000","last_undegraded":"2024-09-21T02:06:44.258053+0000","last_fullsized":"2024-09-21T02:06:44.258053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'168","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.554399+0000","last_change":"2024-09-21T02:04:45.212987+0000","last_active":"2024-09-21T02:06:44.554399+0000","last_peered":"2024-09-21T02:06:44.554399+0000","last_clean":"2024-09-21T02:06:44.554399+0000","last_became_active":"2024-09-21T02:04:45.212611+0000","last_became_peered":"2024-09-21T02:04:45.212611+0000","last_unstale":"2024-09-21T02:06:44.554399+0000","last_undegraded":"2024-09-21T02:06:44.554399+0000","last_fullsized":"2024-09-21T02:06:44.554399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:54.964291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'169","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490796+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:06:44.490796+0000","last_peered":"2024-09-21T02:06:44.490796+0000","last_clean":"2024-09-21T02:06:44.490796+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:06:44.490796+0000","last_undegraded":"2024-09-21T02:06:44.490796+0000","last_fullsized":"2024-09-21T02:06:44.490796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'198","reported_seq":216,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.513284+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:06:44.513284+0000","last_peered":"2024-09-21T02:06:44.513284+0000","last_clean":"2024-09-21T02:06:44.513284+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:06:44.513284+0000","last_undegraded":"2024-09-21T02:06:44.513284+0000","last_fullsized":"2024-09-21T02:06:44.513284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.548720+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:06:44.548720+0000","last_peered":"2024-09-21T02:06:44.548720+0000","last_clean":"2024-09-21T02:06:44.548720+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:06:44.548720+0000","last_undegraded":"2024-09-21T02:06:44.548720+0000","last_fullsized":"2024-09-21T02:06:44.548720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'143","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.258179+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:06:44.258179+0000","last_peered":"2024-09-21T02:06:44.258179+0000","last_clean":"2024-09-21T02:06:44.258179+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:06:44.258179+0000","last_undegraded":"2024-09-21T02:06:44.258179+0000","last_fullsized":"2024-09-21T02:06:44.258179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'189","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.493538+0000","last_change":"2024-09-21T02:04:45.212961+0000","last_active":"2024-09-21T02:06:44.493538+0000","last_peered":"2024-09-21T02:06:44.493538+0000","last_clean":"2024-09-21T02:06:44.493538+0000","last_became_active":"2024-09-21T02:04:45.212587+0000","last_became_peered":"2024-09-21T02:04:45.212587+0000","last_unstale":"2024-09-21T02:06:44.493538+0000","last_undegraded":"2024-09-21T02:06:44.493538+0000","last_fullsized":"2024-09-21T02:06:44.493538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:12.690374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"23'142","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.253560+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:06:44.253560+0000","last_peered":"2024-09-21T02:06:44.253560+0000","last_clean":"2024-09-21T02:06:44.253560+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:06:44.253560+0000","last_undegraded":"2024-09-21T02:06:44.253560+0000","last_fullsized":"2024-09-21T02:06:44.253560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'201","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.533311+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:06:44.533311+0000","last_peered":"2024-09-21T02:06:44.533311+0000","last_clean":"2024-09-21T02:06:44.533311+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:06:44.533311+0000","last_undegraded":"2024-09-21T02:06:44.533311+0000","last_fullsized":"2024-09-21T02:06:44.533311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.262090+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:06:44.262090+0000","last_peered":"2024-09-21T02:06:44.262090+0000","last_clean":"2024-09-21T02:06:44.262090+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:06:44.262090+0000","last_undegraded":"2024-09-21T02:06:44.262090+0000","last_fullsized":"2024-09-21T02:06:44.262090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'202","reported_seq":220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.221781+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:06:44.221781+0000","last_peered":"2024-09-21T02:06:44.221781+0000","last_clean":"2024-09-21T02:06:44.221781+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:06:44.221781+0000","last_undegraded":"2024-09-21T02:06:44.221781+0000","last_fullsized":"2024-09-21T02:06:44.221781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.273171+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:06:44.273171+0000","last_peered":"2024-09-21T02:06:44.273171+0000","last_clean":"2024-09-21T02:06:44.273171+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:06:44.273171+0000","last_undegraded":"2024-09-21T02:06:44.273171+0000","last_fullsized":"2024-09-21T02:06:44.273171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.236933+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:06:44.236933+0000","last_peered":"2024-09-21T02:06:44.236933+0000","last_clean":"2024-09-21T02:06:44.236933+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:06:44.236933+0000","last_undegraded":"2024-09-21T02:06:44.236933+0000","last_fullsized":"2024-09-21T02:06:44.236933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'127","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.232151+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:06:44.232151+0000","last_peered":"2024-09-21T02:06:44.232151+0000","last_clean":"2024-09-21T02:06:44.232151+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:06:44.232151+0000","last_undegraded":"2024-09-21T02:06:44.232151+0000","last_fullsized":"2024-09-21T02:06:44.232151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'145","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.228888+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:06:44.228888+0000","last_peered":"2024-09-21T02:06:44.228888+0000","last_clean":"2024-09-21T02:06:44.228888+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:06:44.228888+0000","last_undegraded":"2024-09-21T02:06:44.228888+0000","last_fullsized":"2024-09-21T02:06:44.228888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'146","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.226313+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:06:44.226313+0000","last_peered":"2024-09-21T02:06:44.226313+0000","last_clean":"2024-09-21T02:06:44.226313+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:06:44.226313+0000","last_undegraded":"2024-09-21T02:06:44.226313+0000","last_fullsized":"2024-09-21T02:06:44.226313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.506925+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:06:44.506925+0000","last_peered":"2024-09-21T02:06:44.506925+0000","last_clean":"2024-09-21T02:06:44.506925+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:06:44.506925+0000","last_undegraded":"2024-09-21T02:06:44.506925+0000","last_fullsized":"2024-09-21T02:06:44.506925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'142","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.255204+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:06:44.255204+0000","last_peered":"2024-09-21T02:06:44.255204+0000","last_clean":"2024-09-21T02:06:44.255204+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:06:44.255204+0000","last_undegraded":"2024-09-21T02:06:44.255204+0000","last_fullsized":"2024-09-21T02:06:44.255204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1643","reported_seq":1663,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:48.101923+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:06:48.101923+0000","last_peered":"2024-09-21T02:06:48.101923+0000","last_clean":"2024-09-21T02:06:48.101923+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:06:48.101923+0000","last_undegraded":"2024-09-21T02:06:48.101923+0000","last_fullsized":"2024-09-21T02:06:48.101923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1643,"log_dups_size":0,"ondisk_log_size":1643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3424840,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1640,"num_write_kb":4925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.218892+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:06:44.218892+0000","last_peered":"2024-09-21T02:06:44.218892+0000","last_clean":"2024-09-21T02:06:44.218892+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:06:44.218892+0000","last_undegraded":"2024-09-21T02:06:44.218892+0000","last_fullsized":"2024-09-21T02:06:44.218892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.279847+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:04:45.279847+0000","last_peered":"2024-09-21T02:04:45.279847+0000","last_clean":"2024-09-21T02:04:45.279847+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:04:45.279847+0000","last_undegraded":"2024-09-21T02:04:45.279847+0000","last_fullsized":"2024-09-21T02:04:45.279847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280546+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:04:45.280546+0000","last_peered":"2024-09-21T02:04:45.280546+0000","last_clean":"2024-09-21T02:04:45.280546+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:04:45.280546+0000","last_undegraded":"2024-09-21T02:04:45.280546+0000","last_fullsized":"2024-09-21T02:04:45.280546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'182","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.493437+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:06:44.493437+0000","last_peered":"2024-09-21T02:06:44.493437+0000","last_clean":"2024-09-21T02:06:44.493437+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:06:44.493437+0000","last_undegraded":"2024-09-21T02:06:44.493437+0000","last_fullsized":"2024-09-21T02:06:44.493437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205335+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:04:45.205335+0000","last_peered":"2024-09-21T02:04:45.205335+0000","last_clean":"2024-09-21T02:04:45.205335+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:04:45.205335+0000","last_undegraded":"2024-09-21T02:04:45.205335+0000","last_fullsized":"2024-09-21T02:04:45.205335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.504661+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:06:44.504661+0000","last_peered":"2024-09-21T02:06:44.504661+0000","last_clean":"2024-09-21T02:06:44.504661+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:06:44.504661+0000","last_undegraded":"2024-09-21T02:06:44.504661+0000","last_fullsized":"2024-09-21T02:06:44.504661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.142553+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:05:47.142553+0000","last_peered":"2024-09-21T02:05:47.142553+0000","last_clean":"2024-09-21T02:05:47.142553+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:05:47.142553+0000","last_undegraded":"2024-09-21T02:05:47.142553+0000","last_fullsized":"2024-09-21T02:05:47.142553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'241","reported_seq":253,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.541162+0000","last_change":"2024-09-21T02:04:45.212312+0000","last_active":"2024-09-21T02:06:44.541162+0000","last_peered":"2024-09-21T02:06:44.541162+0000","last_clean":"2024-09-21T02:06:44.541162+0000","last_became_active":"2024-09-21T02:04:45.211981+0000","last_became_peered":"2024-09-21T02:04:45.211981+0000","last_unstale":"2024-09-21T02:06:44.541162+0000","last_undegraded":"2024-09-21T02:06:44.541162+0000","last_fullsized":"2024-09-21T02:06:44.541162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.148945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205736+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:04:45.205736+0000","last_peered":"2024-09-21T02:04:45.205736+0000","last_clean":"2024-09-21T02:04:45.205736+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:04:45.205736+0000","last_undegraded":"2024-09-21T02:04:45.205736+0000","last_fullsized":"2024-09-21T02:04:45.205736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'151","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.264984+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:06:44.264984+0000","last_peered":"2024-09-21T02:06:44.264984+0000","last_clean":"2024-09-21T02:06:44.264984+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:06:44.264984+0000","last_undegraded":"2024-09-21T02:06:44.264984+0000","last_fullsized":"2024-09-21T02:06:44.264984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:45.329388+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:06:45.329388+0000","last_peered":"2024-09-21T02:06:45.329388+0000","last_clean":"2024-09-21T02:06:45.329388+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:06:45.329388+0000","last_undegraded":"2024-09-21T02:06:45.329388+0000","last_fullsized":"2024-09-21T02:06:45.329388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.231890+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:06:44.231890+0000","last_peered":"2024-09-21T02:06:44.231890+0000","last_clean":"2024-09-21T02:06:44.231890+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:06:44.231890+0000","last_undegraded":"2024-09-21T02:06:44.231890+0000","last_fullsized":"2024-09-21T02:06:44.231890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'2017","reported_seq":2037,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:19.585213+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:06:19.585213+0000","last_peered":"2024-09-21T02:06:19.585213+0000","last_clean":"2024-09-21T02:06:19.585213+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:06:19.585213+0000","last_undegraded":"2024-09-21T02:06:19.585213+0000","last_fullsized":"2024-09-21T02:06:19.585213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2017,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'170","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.488411+0000","last_change":"2024-09-21T02:04:45.213511+0000","last_active":"2024-09-21T02:06:44.488411+0000","last_peered":"2024-09-21T02:06:44.488411+0000","last_clean":"2024-09-21T02:06:44.488411+0000","last_became_active":"2024-09-21T02:04:45.213182+0000","last_became_peered":"2024-09-21T02:04:45.213182+0000","last_unstale":"2024-09-21T02:06:44.488411+0000","last_undegraded":"2024-09-21T02:06:44.488411+0000","last_fullsized":"2024-09-21T02:06:44.488411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:11.613618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280485+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:45.280485+0000","last_peered":"2024-09-21T02:04:45.280485+0000","last_clean":"2024-09-21T02:04:45.280485+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:45.280485+0000","last_undegraded":"2024-09-21T02:04:45.280485+0000","last_fullsized":"2024-09-21T02:04:45.280485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.221865+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:06:44.221865+0000","last_peered":"2024-09-21T02:06:44.221865+0000","last_clean":"2024-09-21T02:06:44.221865+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:06:44.221865+0000","last_undegraded":"2024-09-21T02:06:44.221865+0000","last_fullsized":"2024-09-21T02:06:44.221865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205637+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:04:45.205637+0000","last_peered":"2024-09-21T02:04:45.205637+0000","last_clean":"2024-09-21T02:04:45.205637+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:04:45.205637+0000","last_undegraded":"2024-09-21T02:04:45.205637+0000","last_fullsized":"2024-09-21T02:04:45.205637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'186","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.503215+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:06:44.503215+0000","last_peered":"2024-09-21T02:06:44.503215+0000","last_clean":"2024-09-21T02:06:44.503215+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:06:44.503215+0000","last_undegraded":"2024-09-21T02:06:44.503215+0000","last_fullsized":"2024-09-21T02:06:44.503215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.279800+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:04:45.279800+0000","last_peered":"2024-09-21T02:04:45.279800+0000","last_clean":"2024-09-21T02:04:45.279800+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:04:45.279800+0000","last_undegraded":"2024-09-21T02:04:45.279800+0000","last_fullsized":"2024-09-21T02:04:45.279800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'191","reported_seq":209,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.517881+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:06:44.517881+0000","last_peered":"2024-09-21T02:06:44.517881+0000","last_clean":"2024-09-21T02:06:44.517881+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:06:44.517881+0000","last_undegraded":"2024-09-21T02:06:44.517881+0000","last_fullsized":"2024-09-21T02:06:44.517881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'16","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:16.689517+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:06:16.689517+0000","last_peered":"2024-09-21T02:06:16.689517+0000","last_clean":"2024-09-21T02:06:16.689517+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:06:16.689517+0000","last_undegraded":"2024-09-21T02:06:16.689517+0000","last_fullsized":"2024-09-21T02:06:16.689517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'209","reported_seq":227,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.230309+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:06:44.230309+0000","last_peered":"2024-09-21T02:06:44.230309+0000","last_clean":"2024-09-21T02:06:44.230309+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:06:44.230309+0000","last_undegraded":"2024-09-21T02:06:44.230309+0000","last_fullsized":"2024-09-21T02:06:44.230309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.203910+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:04:45.203910+0000","last_peered":"2024-09-21T02:04:45.203910+0000","last_clean":"2024-09-21T02:04:45.203910+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:04:45.203910+0000","last_undegraded":"2024-09-21T02:04:45.203910+0000","last_fullsized":"2024-09-21T02:04:45.203910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.566471+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:06:44.566471+0000","last_peered":"2024-09-21T02:06:44.566471+0000","last_clean":"2024-09-21T02:06:44.566471+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:06:44.566471+0000","last_undegraded":"2024-09-21T02:06:44.566471+0000","last_fullsized":"2024-09-21T02:06:44.566471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208099+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:45.208099+0000","last_peered":"2024-09-21T02:04:45.208099+0000","last_clean":"2024-09-21T02:04:45.208099+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:45.208099+0000","last_undegraded":"2024-09-21T02:04:45.208099+0000","last_fullsized":"2024-09-21T02:04:45.208099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.212614+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:06:44.212614+0000","last_peered":"2024-09-21T02:06:44.212614+0000","last_clean":"2024-09-21T02:06:44.212614+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:06:44.212614+0000","last_undegraded":"2024-09-21T02:06:44.212614+0000","last_fullsized":"2024-09-21T02:06:44.212614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208098+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:04:45.208098+0000","last_peered":"2024-09-21T02:04:45.208098+0000","last_clean":"2024-09-21T02:04:45.208098+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:04:45.208098+0000","last_undegraded":"2024-09-21T02:04:45.208098+0000","last_fullsized":"2024-09-21T02:04:45.208098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'159","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.513761+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:06:44.513761+0000","last_peered":"2024-09-21T02:06:44.513761+0000","last_clean":"2024-09-21T02:06:44.513761+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:06:44.513761+0000","last_undegraded":"2024-09-21T02:06:44.513761+0000","last_fullsized":"2024-09-21T02:06:44.513761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207953+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:04:45.207953+0000","last_peered":"2024-09-21T02:04:45.207953+0000","last_clean":"2024-09-21T02:04:45.207953+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:04:45.207953+0000","last_undegraded":"2024-09-21T02:04:45.207953+0000","last_fullsized":"2024-09-21T02:04:45.207953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.270827+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:06:44.270827+0000","last_peered":"2024-09-21T02:06:44.270827+0000","last_clean":"2024-09-21T02:06:44.270827+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:06:44.270827+0000","last_undegraded":"2024-09-21T02:06:44.270827+0000","last_fullsized":"2024-09-21T02:06:44.270827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211129+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:04:45.211129+0000","last_peered":"2024-09-21T02:04:45.211129+0000","last_clean":"2024-09-21T02:04:45.211129+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:04:45.211129+0000","last_undegraded":"2024-09-21T02:04:45.211129+0000","last_fullsized":"2024-09-21T02:04:45.211129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205332+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:04:45.205332+0000","last_peered":"2024-09-21T02:04:45.205332+0000","last_clean":"2024-09-21T02:04:45.205332+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:04:45.205332+0000","last_undegraded":"2024-09-21T02:04:45.205332+0000","last_fullsized":"2024-09-21T02:04:45.205332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.211254+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:06:44.211254+0000","last_peered":"2024-09-21T02:06:44.211254+0000","last_clean":"2024-09-21T02:06:44.211254+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:06:44.211254+0000","last_undegraded":"2024-09-21T02:06:44.211254+0000","last_fullsized":"2024-09-21T02:06:44.211254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210923+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:04:45.210923+0000","last_peered":"2024-09-21T02:04:45.210923+0000","last_clean":"2024-09-21T02:04:45.210923+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:04:45.210923+0000","last_undegraded":"2024-09-21T02:04:45.210923+0000","last_fullsized":"2024-09-21T02:04:45.210923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.562728+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:06:44.562728+0000","last_peered":"2024-09-21T02:06:44.562728+0000","last_clean":"2024-09-21T02:06:44.562728+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:06:44.562728+0000","last_undegraded":"2024-09-21T02:06:44.562728+0000","last_fullsized":"2024-09-21T02:06:44.562728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:05.466556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'10","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:47.179158+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:06:47.179158+0000","last_peered":"2024-09-21T02:06:47.179158+0000","last_clean":"2024-09-21T02:06:47.179158+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:06:47.179158+0000","last_undegraded":"2024-09-21T02:06:47.179158+0000","last_fullsized":"2024-09-21T02:06:47.179158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":595,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.494702+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:06:44.494702+0000","last_peered":"2024-09-21T02:06:44.494702+0000","last_clean":"2024-09-21T02:06:44.494702+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:06:44.494702+0000","last_undegraded":"2024-09-21T02:06:44.494702+0000","last_fullsized":"2024-09-21T02:06:44.494702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205672+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:04:45.205672+0000","last_peered":"2024-09-21T02:04:45.205672+0000","last_clean":"2024-09-21T02:04:45.205672+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:04:45.205672+0000","last_undegraded":"2024-09-21T02:04:45.205672+0000","last_fullsized":"2024-09-21T02:04:45.205672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.555314+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:06:44.555314+0000","last_peered":"2024-09-21T02:06:44.555314+0000","last_clean":"2024-09-21T02:06:44.555314+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:06:44.555314+0000","last_undegraded":"2024-09-21T02:06:44.555314+0000","last_fullsized":"2024-09-21T02:06:44.555314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207553+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:04:45.207553+0000","last_peered":"2024-09-21T02:04:45.207553+0000","last_clean":"2024-09-21T02:04:45.207553+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:04:45.207553+0000","last_undegraded":"2024-09-21T02:04:45.207553+0000","last_fullsized":"2024-09-21T02:04:45.207553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'155","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.258318+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:06:44.258318+0000","last_peered":"2024-09-21T02:06:44.258318+0000","last_clean":"2024-09-21T02:06:44.258318+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:06:44.258318+0000","last_undegraded":"2024-09-21T02:06:44.258318+0000","last_fullsized":"2024-09-21T02:06:44.258318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211858+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:04:45.211858+0000","last_peered":"2024-09-21T02:04:45.211858+0000","last_clean":"2024-09-21T02:04:45.211858+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:04:45.211858+0000","last_undegraded":"2024-09-21T02:04:45.211858+0000","last_fullsized":"2024-09-21T02:04:45.211858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.555740+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:06:44.555740+0000","last_peered":"2024-09-21T02:06:44.555740+0000","last_clean":"2024-09-21T02:06:44.555740+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:06:44.555740+0000","last_undegraded":"2024-09-21T02:06:44.555740+0000","last_fullsized":"2024-09-21T02:06:44.555740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'21","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:43.130438+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:06:43.130438+0000","last_peered":"2024-09-21T02:06:43.130438+0000","last_clean":"2024-09-21T02:06:43.130438+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:06:43.130438+0000","last_undegraded":"2024-09-21T02:06:43.130438+0000","last_fullsized":"2024-09-21T02:06:43.130438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'161","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.495833+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:06:44.495833+0000","last_peered":"2024-09-21T02:06:44.495833+0000","last_clean":"2024-09-21T02:06:44.495833+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:06:44.495833+0000","last_undegraded":"2024-09-21T02:06:44.495833+0000","last_fullsized":"2024-09-21T02:06:44.495833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:09.584145+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:06:09.584145+0000","last_peered":"2024-09-21T02:06:09.584145+0000","last_clean":"2024-09-21T02:06:09.584145+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:06:09.584145+0000","last_undegraded":"2024-09-21T02:06:09.584145+0000","last_fullsized":"2024-09-21T02:06:09.584145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.520907+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:06:44.520907+0000","last_peered":"2024-09-21T02:06:44.520907+0000","last_clean":"2024-09-21T02:06:44.520907+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:06:44.520907+0000","last_undegraded":"2024-09-21T02:06:44.520907+0000","last_fullsized":"2024-09-21T02:06:44.520907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210942+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:04:45.210942+0000","last_peered":"2024-09-21T02:04:45.210942+0000","last_clean":"2024-09-21T02:04:45.210942+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:04:45.210942+0000","last_undegraded":"2024-09-21T02:04:45.210942+0000","last_fullsized":"2024-09-21T02:04:45.210942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.558277+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:06:44.558277+0000","last_peered":"2024-09-21T02:06:44.558277+0000","last_clean":"2024-09-21T02:06:44.558277+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:06:44.558277+0000","last_undegraded":"2024-09-21T02:06:44.558277+0000","last_fullsized":"2024-09-21T02:06:44.558277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207751+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:04:45.207751+0000","last_peered":"2024-09-21T02:04:45.207751+0000","last_clean":"2024-09-21T02:04:45.207751+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:04:45.207751+0000","last_undegraded":"2024-09-21T02:04:45.207751+0000","last_fullsized":"2024-09-21T02:04:45.207751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.214359+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:06:44.214359+0000","last_peered":"2024-09-21T02:06:44.214359+0000","last_clean":"2024-09-21T02:06:44.214359+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:06:44.214359+0000","last_undegraded":"2024-09-21T02:06:44.214359+0000","last_fullsized":"2024-09-21T02:06:44.214359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207926+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:04:45.207926+0000","last_peered":"2024-09-21T02:04:45.207926+0000","last_clean":"2024-09-21T02:04:45.207926+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:04:45.207926+0000","last_undegraded":"2024-09-21T02:04:45.207926+0000","last_fullsized":"2024-09-21T02:04:45.207926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'156","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.499249+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:06:44.499249+0000","last_peered":"2024-09-21T02:06:44.499249+0000","last_clean":"2024-09-21T02:06:44.499249+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:06:44.499249+0000","last_undegraded":"2024-09-21T02:06:44.499249+0000","last_fullsized":"2024-09-21T02:06:44.499249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.141927+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:05:47.141927+0000","last_peered":"2024-09-21T02:05:47.141927+0000","last_clean":"2024-09-21T02:05:47.141927+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:05:47.141927+0000","last_undegraded":"2024-09-21T02:05:47.141927+0000","last_fullsized":"2024-09-21T02:05:47.141927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.537214+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:06:44.537214+0000","last_peered":"2024-09-21T02:06:44.537214+0000","last_clean":"2024-09-21T02:06:44.537214+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:06:44.537214+0000","last_undegraded":"2024-09-21T02:06:44.537214+0000","last_fullsized":"2024-09-21T02:06:44.537214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'190","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.515015+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:06:44.515015+0000","last_peered":"2024-09-21T02:06:44.515015+0000","last_clean":"2024-09-21T02:06:44.515015+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:06:44.515015+0000","last_undegraded":"2024-09-21T02:06:44.515015+0000","last_fullsized":"2024-09-21T02:06:44.515015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211388+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:45.211388+0000","last_peered":"2024-09-21T02:04:45.211388+0000","last_clean":"2024-09-21T02:04:45.211388+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:45.211388+0000","last_undegraded":"2024-09-21T02:04:45.211388+0000","last_fullsized":"2024-09-21T02:04:45.211388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.487942+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:06:44.487942+0000","last_peered":"2024-09-21T02:06:44.487942+0000","last_clean":"2024-09-21T02:06:44.487942+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:06:44.487942+0000","last_undegraded":"2024-09-21T02:06:44.487942+0000","last_fullsized":"2024-09-21T02:06:44.487942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:08.902449+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:06:08.902449+0000","last_peered":"2024-09-21T02:06:08.902449+0000","last_clean":"2024-09-21T02:06:08.902449+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:06:08.902449+0000","last_undegraded":"2024-09-21T02:06:08.902449+0000","last_fullsized":"2024-09-21T02:06:08.902449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.244570+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:06:44.244570+0000","last_peered":"2024-09-21T02:06:44.244570+0000","last_clean":"2024-09-21T02:06:44.244570+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:06:44.244570+0000","last_undegraded":"2024-09-21T02:06:44.244570+0000","last_fullsized":"2024-09-21T02:06:44.244570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280872+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:04:45.280872+0000","last_peered":"2024-09-21T02:04:45.280872+0000","last_clean":"2024-09-21T02:04:45.280872+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:04:45.280872+0000","last_undegraded":"2024-09-21T02:04:45.280872+0000","last_fullsized":"2024-09-21T02:04:45.280872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490694+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:06:44.490694+0000","last_peered":"2024-09-21T02:06:44.490694+0000","last_clean":"2024-09-21T02:06:44.490694+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:06:44.490694+0000","last_undegraded":"2024-09-21T02:06:44.490694+0000","last_fullsized":"2024-09-21T02:06:44.490694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280763+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:04:45.280763+0000","last_peered":"2024-09-21T02:04:45.280763+0000","last_clean":"2024-09-21T02:04:45.280763+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:04:45.280763+0000","last_undegraded":"2024-09-21T02:04:45.280763+0000","last_fullsized":"2024-09-21T02:04:45.280763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7623565,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3817,"num_write_kb":11013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15400960,"data_stored":15247130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3793,"ondisk_log_size":3793,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57564,"kb_used_data":4172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314276,"statfs":{"total":96636764160,"available":96577818624,"internally_reserved":0,"allocated":4272128,"data_stored":4100839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":86315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":85907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83228,"kb_used_data":7692,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94288612,"statfs":{"total":96636764160,"available":96551538688,"internally_reserved":0,"allocated":7876608,"data_stored":7705348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":86260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":85881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53572,"kb_used_data":4340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318268,"statfs":{"total":96636764160,"available":96581906432,"internally_reserved":0,"allocated":4444160,"data_stored":4280847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":843776,"data_stored":675965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7643136,"data_stored":7620411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3444736,"data_stored":3425534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:07:16.859 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:07:16.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.007+0000 7f5ee4c71640 1 -- 172.21.15.88:0/4059652274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 msgr2=0x7f5ee0156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.007+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/4059652274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0156df0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8005fd0 tx=0x7f5ec802a5c0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.008+0000 7f5ee4c71640 1 -- 172.21.15.88:0/4059652274 shutdown_connections 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.008+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/4059652274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ee0070fd0 0x7f5ee00713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.008+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/4059652274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0156df0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.008+0000 7f5ee4c71640 1 -- 172.21.15.88:0/4059652274 >> 172.21.15.88:0/4059652274 conn(0x7f5ee00729d0 msgr2=0x7f5ee0072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.008+0000 7f5ee4c71640 1 -- 172.21.15.88:0/4059652274 shutdown_connections 2024-09-21T02:07:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.008+0000 7f5ee4c71640 1 -- 172.21.15.88:0/4059652274 wait complete. 2024-09-21T02:07:17.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.010+0000 7f5ee4c71640 1 Processor -- start 2024-09-21T02:07:17.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.010+0000 7f5ee4c71640 1 -- start start 2024-09-21T02:07:17.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ee4c71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ee0070fd0 0x7f5ee01efde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ee4c71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ee00c3fc0 0x7f5ee00cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ee4c71640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ee4c71640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0071920 con 0x7f5ee00c8a00 2024-09-21T02:07:17.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ee4c71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0070ab0 con 0x7f5ee0070fd0 2024-09-21T02:07:17.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ee4c71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee015a5b0 con 0x7f5ee00c3fc0 2024-09-21T02:07:17.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39582/0 (socket says 172.21.15.88:39582) 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 -- 172.21.15.88:0/919778955 learned_addr learned my addr 172.21.15.88:0/919778955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ede59c640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ee0070fd0 0x7f5ee01efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5ed5d9b640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ee00c3fc0 0x7f5ee00cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 -- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ee00c3fc0 msgr2=0x7f5ee00cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ee00c3fc0 0x7f5ee00cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 -- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ee0070fd0 msgr2=0x7f5ee01efde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ee0070fd0 0x7f5ee01efde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 -- 172.21.15.88:0/919778955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee01f0320 con 0x7f5ee00c8a00 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.011+0000 7f5eded9d640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0150050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800ba10 tx=0x7f5ed800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.012+0000 7f5ed77fe640 1 -- 172.21.15.88:0/919778955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8019070 con 0x7f5ee00c8a00 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.012+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee01ef8e0 con 0x7f5ee00c8a00 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.012+0000 7f5ed77fe640 1 -- 172.21.15.88:0/919778955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ed800cc50 con 0x7f5ee00c8a00 2024-09-21T02:07:17.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.012+0000 7f5ed77fe640 1 -- 172.21.15.88:0/919778955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed80146a0 con 0x7f5ee00c8a00 2024-09-21T02:07:17.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.012+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee00cf140 con 0x7f5ee00c8a00 2024-09-21T02:07:17.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.013+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee00c8f30 con 0x7f5ee00c8a00 2024-09-21T02:07:17.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.014+0000 7f5ed77fe640 1 -- 172.21.15.88:0/919778955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ed801c070 con 0x7f5ee00c8a00 2024-09-21T02:07:17.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.014+0000 7f5ed77fe640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5ec0078b90 0x7f5ec007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.014+0000 7f5ed77fe640 1 -- 172.21.15.88:0/919778955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5ed809aae0 con 0x7f5ee00c8a00 2024-09-21T02:07:17.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.014+0000 7f5ede59c640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5ec0078b90 0x7f5ec007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.015+0000 7f5ede59c640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5ec0078b90 0x7f5ec007b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0006fd0 tx=0x7f5ed0009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:17.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.019+0000 7f5ed77fe640 1 -- 172.21.15.88:0/919778955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ed8062860 con 0x7f5ee00c8a00 2024-09-21T02:07:17.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.197+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ee0072360 con 0x7f5ec0078b90 2024-09-21T02:07:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.201+0000 7f5ed77fe640 1 -- 172.21.15.88:0/919778955 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253022 (secure 0 0 0) 0x7f5ee0072360 con 0x7f5ec0078b90 2024-09-21T02:07:17.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:17.205 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5ec0078b90 msgr2=0x7f5ec007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5ec0078b90 0x7f5ec007b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0006fd0 tx=0x7f5ed0009060 comp rx=0 tx=0).stop 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 msgr2=0x7f5ee0150050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0150050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800ba10 tx=0x7f5ed800bed0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 shutdown_connections 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5ec0078b90 0x7f5ec007b0a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee00c8a00 0x7f5ee0150050 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ee00c3fc0 0x7f5ee00cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 --2- 172.21.15.88:0/919778955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ee0070fd0 0x7f5ee01efde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 >> 172.21.15.88:0/919778955 conn(0x7f5ee00729d0 msgr2=0x7f5ee0158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 shutdown_connections 2024-09-21T02:07:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.204+0000 7f5ee4c71640 1 -- 172.21.15.88:0/919778955 wait complete. 2024-09-21T02:07:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-21T02:07:16.874795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8213933,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":13137,"num_write_kb":25178466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14851,"ondisk_log_size":14851,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347944,"kb_used_data":18024,"kb_used_omap":12,"kb_used_meta":329779,"kb_avail":754626776,"statfs":{"total":773094113280,"available":772737818624,"internally_reserved":0,"allocated":18456576,"data_stored":17107503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":337694258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003371"},"pg_stats":[{"pgid":"3.3f","version":"23'167","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.930261+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:07:15.930261+0000","last_peered":"2024-09-21T02:07:15.930261+0000","last_clean":"2024-09-21T02:07:15.930261+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:07:15.930261+0000","last_undegraded":"2024-09-21T02:07:15.930261+0000","last_fullsized":"2024-09-21T02:07:15.930261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.266433+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:06:44.266433+0000","last_peered":"2024-09-21T02:06:44.266433+0000","last_clean":"2024-09-21T02:06:44.266433+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:06:44.266433+0000","last_undegraded":"2024-09-21T02:06:44.266433+0000","last_fullsized":"2024-09-21T02:06:44.266433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.550328+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:06:44.550328+0000","last_peered":"2024-09-21T02:06:44.550328+0000","last_clean":"2024-09-21T02:06:44.550328+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:06:44.550328+0000","last_undegraded":"2024-09-21T02:06:44.550328+0000","last_fullsized":"2024-09-21T02:06:44.550328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'193","reported_seq":211,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934380+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:07:15.934380+0000","last_peered":"2024-09-21T02:07:15.934380+0000","last_clean":"2024-09-21T02:07:15.934380+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:07:15.934380+0000","last_undegraded":"2024-09-21T02:07:15.934380+0000","last_fullsized":"2024-09-21T02:07:15.934380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'174","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.924744+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:07:15.924744+0000","last_peered":"2024-09-21T02:07:15.924744+0000","last_clean":"2024-09-21T02:07:15.924744+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:07:15.924744+0000","last_undegraded":"2024-09-21T02:07:15.924744+0000","last_fullsized":"2024-09-21T02:07:15.924744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.920524+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:07:15.920524+0000","last_peered":"2024-09-21T02:07:15.920524+0000","last_clean":"2024-09-21T02:07:15.920524+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:07:15.920524+0000","last_undegraded":"2024-09-21T02:07:15.920524+0000","last_fullsized":"2024-09-21T02:07:15.920524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'188","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.925429+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:07:15.925429+0000","last_peered":"2024-09-21T02:07:15.925429+0000","last_clean":"2024-09-21T02:07:15.925429+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:07:15.925429+0000","last_undegraded":"2024-09-21T02:07:15.925429+0000","last_fullsized":"2024-09-21T02:07:15.925429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'195","reported_seq":213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.539730+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:06:44.539730+0000","last_peered":"2024-09-21T02:06:44.539730+0000","last_clean":"2024-09-21T02:06:44.539730+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:06:44.539730+0000","last_undegraded":"2024-09-21T02:06:44.539730+0000","last_fullsized":"2024-09-21T02:06:44.539730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'178","reported_seq":196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936803+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:07:15.936803+0000","last_peered":"2024-09-21T02:07:15.936803+0000","last_clean":"2024-09-21T02:07:15.936803+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:07:15.936803+0000","last_undegraded":"2024-09-21T02:07:15.936803+0000","last_fullsized":"2024-09-21T02:07:15.936803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.500726+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:06:44.500726+0000","last_peered":"2024-09-21T02:06:44.500726+0000","last_clean":"2024-09-21T02:06:44.500726+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:06:44.500726+0000","last_undegraded":"2024-09-21T02:06:44.500726+0000","last_fullsized":"2024-09-21T02:06:44.500726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'208","reported_seq":226,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.509048+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:06:44.509048+0000","last_peered":"2024-09-21T02:06:44.509048+0000","last_clean":"2024-09-21T02:06:44.509048+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:06:44.509048+0000","last_undegraded":"2024-09-21T02:06:44.509048+0000","last_fullsized":"2024-09-21T02:06:44.509048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'166","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932783+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:07:15.932783+0000","last_peered":"2024-09-21T02:07:15.932783+0000","last_clean":"2024-09-21T02:07:15.932783+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:07:15.932783+0000","last_undegraded":"2024-09-21T02:07:15.932783+0000","last_fullsized":"2024-09-21T02:07:15.932783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'167","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.479687+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:06:44.479687+0000","last_peered":"2024-09-21T02:06:44.479687+0000","last_clean":"2024-09-21T02:06:44.479687+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:06:44.479687+0000","last_undegraded":"2024-09-21T02:06:44.479687+0000","last_fullsized":"2024-09-21T02:06:44.479687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.218722+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:06:44.218722+0000","last_peered":"2024-09-21T02:06:44.218722+0000","last_clean":"2024-09-21T02:06:44.218722+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:06:44.218722+0000","last_undegraded":"2024-09-21T02:06:44.218722+0000","last_fullsized":"2024-09-21T02:06:44.218722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'170","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490826+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:06:44.490826+0000","last_peered":"2024-09-21T02:06:44.490826+0000","last_clean":"2024-09-21T02:06:44.490826+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:06:44.490826+0000","last_undegraded":"2024-09-21T02:06:44.490826+0000","last_fullsized":"2024-09-21T02:06:44.490826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.921029+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:07:15.921029+0000","last_peered":"2024-09-21T02:07:15.921029+0000","last_clean":"2024-09-21T02:07:15.921029+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:07:15.921029+0000","last_undegraded":"2024-09-21T02:07:15.921029+0000","last_fullsized":"2024-09-21T02:07:15.921029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'176","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.929050+0000","last_change":"2024-09-21T02:04:45.212987+0000","last_active":"2024-09-21T02:07:15.929050+0000","last_peered":"2024-09-21T02:07:15.929050+0000","last_clean":"2024-09-21T02:07:15.929050+0000","last_became_active":"2024-09-21T02:04:45.212611+0000","last_became_peered":"2024-09-21T02:04:45.212611+0000","last_unstale":"2024-09-21T02:07:15.929050+0000","last_undegraded":"2024-09-21T02:07:15.929050+0000","last_fullsized":"2024-09-21T02:07:15.929050+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:54.964291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'179","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.902990+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:07:15.902990+0000","last_peered":"2024-09-21T02:07:15.902990+0000","last_clean":"2024-09-21T02:07:15.902990+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:07:15.902990+0000","last_undegraded":"2024-09-21T02:07:15.902990+0000","last_fullsized":"2024-09-21T02:07:15.902990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'205","reported_seq":223,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.927156+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:07:15.927156+0000","last_peered":"2024-09-21T02:07:15.927156+0000","last_clean":"2024-09-21T02:07:15.927156+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:07:15.927156+0000","last_undegraded":"2024-09-21T02:07:15.927156+0000","last_fullsized":"2024-09-21T02:07:15.927156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.548720+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:06:44.548720+0000","last_peered":"2024-09-21T02:06:44.548720+0000","last_clean":"2024-09-21T02:06:44.548720+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:06:44.548720+0000","last_undegraded":"2024-09-21T02:06:44.548720+0000","last_fullsized":"2024-09-21T02:06:44.548720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'143","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.258179+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:06:44.258179+0000","last_peered":"2024-09-21T02:06:44.258179+0000","last_clean":"2024-09-21T02:06:44.258179+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:06:44.258179+0000","last_undegraded":"2024-09-21T02:06:44.258179+0000","last_fullsized":"2024-09-21T02:06:44.258179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'189","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.493538+0000","last_change":"2024-09-21T02:04:45.212961+0000","last_active":"2024-09-21T02:06:44.493538+0000","last_peered":"2024-09-21T02:06:44.493538+0000","last_clean":"2024-09-21T02:06:44.493538+0000","last_became_active":"2024-09-21T02:04:45.212587+0000","last_became_peered":"2024-09-21T02:04:45.212587+0000","last_unstale":"2024-09-21T02:06:44.493538+0000","last_undegraded":"2024-09-21T02:06:44.493538+0000","last_fullsized":"2024-09-21T02:06:44.493538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:12.690374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"23'142","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.253560+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:06:44.253560+0000","last_peered":"2024-09-21T02:06:44.253560+0000","last_clean":"2024-09-21T02:06:44.253560+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:06:44.253560+0000","last_undegraded":"2024-09-21T02:06:44.253560+0000","last_fullsized":"2024-09-21T02:06:44.253560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'207","reported_seq":225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.935036+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:07:15.935036+0000","last_peered":"2024-09-21T02:07:15.935036+0000","last_clean":"2024-09-21T02:07:15.935036+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:07:15.935036+0000","last_undegraded":"2024-09-21T02:07:15.935036+0000","last_fullsized":"2024-09-21T02:07:15.935036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.262090+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:06:44.262090+0000","last_peered":"2024-09-21T02:06:44.262090+0000","last_clean":"2024-09-21T02:06:44.262090+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:06:44.262090+0000","last_undegraded":"2024-09-21T02:06:44.262090+0000","last_fullsized":"2024-09-21T02:06:44.262090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'202","reported_seq":220,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.221781+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:06:44.221781+0000","last_peered":"2024-09-21T02:06:44.221781+0000","last_clean":"2024-09-21T02:06:44.221781+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:06:44.221781+0000","last_undegraded":"2024-09-21T02:06:44.221781+0000","last_fullsized":"2024-09-21T02:06:44.221781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.273171+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:06:44.273171+0000","last_peered":"2024-09-21T02:06:44.273171+0000","last_clean":"2024-09-21T02:06:44.273171+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:06:44.273171+0000","last_undegraded":"2024-09-21T02:06:44.273171+0000","last_fullsized":"2024-09-21T02:06:44.273171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'148","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.236933+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:06:44.236933+0000","last_peered":"2024-09-21T02:06:44.236933+0000","last_clean":"2024-09-21T02:06:44.236933+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:06:44.236933+0000","last_undegraded":"2024-09-21T02:06:44.236933+0000","last_fullsized":"2024-09-21T02:06:44.236933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'127","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.232151+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:06:44.232151+0000","last_peered":"2024-09-21T02:06:44.232151+0000","last_clean":"2024-09-21T02:06:44.232151+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:06:44.232151+0000","last_undegraded":"2024-09-21T02:06:44.232151+0000","last_fullsized":"2024-09-21T02:06:44.232151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932066+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:07:15.932066+0000","last_peered":"2024-09-21T02:07:15.932066+0000","last_clean":"2024-09-21T02:07:15.932066+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:07:15.932066+0000","last_undegraded":"2024-09-21T02:07:15.932066+0000","last_fullsized":"2024-09-21T02:07:15.932066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.904301+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:07:15.904301+0000","last_peered":"2024-09-21T02:07:15.904301+0000","last_clean":"2024-09-21T02:07:15.904301+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:07:15.904301+0000","last_undegraded":"2024-09-21T02:07:15.904301+0000","last_fullsized":"2024-09-21T02:07:15.904301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'171","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.506925+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:06:44.506925+0000","last_peered":"2024-09-21T02:06:44.506925+0000","last_clean":"2024-09-21T02:06:44.506925+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:06:44.506925+0000","last_undegraded":"2024-09-21T02:06:44.506925+0000","last_fullsized":"2024-09-21T02:06:44.506925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.910180+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:07:15.910180+0000","last_peered":"2024-09-21T02:07:15.910180+0000","last_clean":"2024-09-21T02:07:15.910180+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:07:15.910180+0000","last_undegraded":"2024-09-21T02:07:15.910180+0000","last_fullsized":"2024-09-21T02:07:15.910180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1643","reported_seq":1663,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:48.101923+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:06:48.101923+0000","last_peered":"2024-09-21T02:06:48.101923+0000","last_clean":"2024-09-21T02:06:48.101923+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:06:48.101923+0000","last_undegraded":"2024-09-21T02:06:48.101923+0000","last_fullsized":"2024-09-21T02:06:48.101923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1643,"log_dups_size":0,"ondisk_log_size":1643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3424840,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1640,"num_write_kb":4925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.218892+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:06:44.218892+0000","last_peered":"2024-09-21T02:06:44.218892+0000","last_clean":"2024-09-21T02:06:44.218892+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:06:44.218892+0000","last_undegraded":"2024-09-21T02:06:44.218892+0000","last_fullsized":"2024-09-21T02:06:44.218892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.279847+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:04:45.279847+0000","last_peered":"2024-09-21T02:04:45.279847+0000","last_clean":"2024-09-21T02:04:45.279847+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:04:45.279847+0000","last_undegraded":"2024-09-21T02:04:45.279847+0000","last_fullsized":"2024-09-21T02:04:45.279847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280546+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:04:45.280546+0000","last_peered":"2024-09-21T02:04:45.280546+0000","last_clean":"2024-09-21T02:04:45.280546+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:04:45.280546+0000","last_undegraded":"2024-09-21T02:04:45.280546+0000","last_fullsized":"2024-09-21T02:04:45.280546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'182","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.493437+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:06:44.493437+0000","last_peered":"2024-09-21T02:06:44.493437+0000","last_clean":"2024-09-21T02:06:44.493437+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:06:44.493437+0000","last_undegraded":"2024-09-21T02:06:44.493437+0000","last_fullsized":"2024-09-21T02:06:44.493437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205335+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:04:45.205335+0000","last_peered":"2024-09-21T02:04:45.205335+0000","last_clean":"2024-09-21T02:04:45.205335+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:04:45.205335+0000","last_undegraded":"2024-09-21T02:04:45.205335+0000","last_fullsized":"2024-09-21T02:04:45.205335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.504661+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:06:44.504661+0000","last_peered":"2024-09-21T02:06:44.504661+0000","last_clean":"2024-09-21T02:06:44.504661+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:06:44.504661+0000","last_undegraded":"2024-09-21T02:06:44.504661+0000","last_fullsized":"2024-09-21T02:06:44.504661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.142553+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:05:47.142553+0000","last_peered":"2024-09-21T02:05:47.142553+0000","last_clean":"2024-09-21T02:05:47.142553+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:05:47.142553+0000","last_undegraded":"2024-09-21T02:05:47.142553+0000","last_fullsized":"2024-09-21T02:05:47.142553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'251","reported_seq":263,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.929179+0000","last_change":"2024-09-21T02:04:45.212312+0000","last_active":"2024-09-21T02:07:15.929179+0000","last_peered":"2024-09-21T02:07:15.929179+0000","last_clean":"2024-09-21T02:07:15.929179+0000","last_became_active":"2024-09-21T02:04:45.211981+0000","last_became_peered":"2024-09-21T02:04:45.211981+0000","last_unstale":"2024-09-21T02:07:15.929179+0000","last_undegraded":"2024-09-21T02:07:15.929179+0000","last_fullsized":"2024-09-21T02:07:15.929179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.148945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205736+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:04:45.205736+0000","last_peered":"2024-09-21T02:04:45.205736+0000","last_clean":"2024-09-21T02:04:45.205736+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:04:45.205736+0000","last_undegraded":"2024-09-21T02:04:45.205736+0000","last_fullsized":"2024-09-21T02:04:45.205736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'151","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.264984+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:06:44.264984+0000","last_peered":"2024-09-21T02:06:44.264984+0000","last_clean":"2024-09-21T02:06:44.264984+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:06:44.264984+0000","last_undegraded":"2024-09-21T02:06:44.264984+0000","last_fullsized":"2024-09-21T02:06:44.264984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:45.329388+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:06:45.329388+0000","last_peered":"2024-09-21T02:06:45.329388+0000","last_clean":"2024-09-21T02:06:45.329388+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:06:45.329388+0000","last_undegraded":"2024-09-21T02:06:45.329388+0000","last_fullsized":"2024-09-21T02:06:45.329388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.937136+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:07:15.937136+0000","last_peered":"2024-09-21T02:07:15.937136+0000","last_clean":"2024-09-21T02:07:15.937136+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:07:15.937136+0000","last_undegraded":"2024-09-21T02:07:15.937136+0000","last_fullsized":"2024-09-21T02:07:15.937136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'2017","reported_seq":2037,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:19.585213+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:06:19.585213+0000","last_peered":"2024-09-21T02:06:19.585213+0000","last_clean":"2024-09-21T02:06:19.585213+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:06:19.585213+0000","last_undegraded":"2024-09-21T02:06:19.585213+0000","last_fullsized":"2024-09-21T02:06:19.585213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2017,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'176","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.927622+0000","last_change":"2024-09-21T02:04:45.213511+0000","last_active":"2024-09-21T02:07:15.927622+0000","last_peered":"2024-09-21T02:07:15.927622+0000","last_clean":"2024-09-21T02:07:15.927622+0000","last_became_active":"2024-09-21T02:04:45.213182+0000","last_became_peered":"2024-09-21T02:04:45.213182+0000","last_unstale":"2024-09-21T02:07:15.927622+0000","last_undegraded":"2024-09-21T02:07:15.927622+0000","last_fullsized":"2024-09-21T02:07:15.927622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:11.613618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280485+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:45.280485+0000","last_peered":"2024-09-21T02:04:45.280485+0000","last_clean":"2024-09-21T02:04:45.280485+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:45.280485+0000","last_undegraded":"2024-09-21T02:04:45.280485+0000","last_fullsized":"2024-09-21T02:04:45.280485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.221865+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:06:44.221865+0000","last_peered":"2024-09-21T02:06:44.221865+0000","last_clean":"2024-09-21T02:06:44.221865+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:06:44.221865+0000","last_undegraded":"2024-09-21T02:06:44.221865+0000","last_fullsized":"2024-09-21T02:06:44.221865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205637+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:04:45.205637+0000","last_peered":"2024-09-21T02:04:45.205637+0000","last_clean":"2024-09-21T02:04:45.205637+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:04:45.205637+0000","last_undegraded":"2024-09-21T02:04:45.205637+0000","last_fullsized":"2024-09-21T02:04:45.205637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'201","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.937095+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:07:15.937095+0000","last_peered":"2024-09-21T02:07:15.937095+0000","last_clean":"2024-09-21T02:07:15.937095+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:07:15.937095+0000","last_undegraded":"2024-09-21T02:07:15.937095+0000","last_fullsized":"2024-09-21T02:07:15.937095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.279800+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:04:45.279800+0000","last_peered":"2024-09-21T02:04:45.279800+0000","last_clean":"2024-09-21T02:04:45.279800+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:04:45.279800+0000","last_undegraded":"2024-09-21T02:04:45.279800+0000","last_fullsized":"2024-09-21T02:04:45.279800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":844,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'191","reported_seq":209,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.517881+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:06:44.517881+0000","last_peered":"2024-09-21T02:06:44.517881+0000","last_clean":"2024-09-21T02:06:44.517881+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:06:44.517881+0000","last_undegraded":"2024-09-21T02:06:44.517881+0000","last_fullsized":"2024-09-21T02:06:44.517881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'16","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:16.689517+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:06:16.689517+0000","last_peered":"2024-09-21T02:06:16.689517+0000","last_clean":"2024-09-21T02:06:16.689517+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:06:16.689517+0000","last_undegraded":"2024-09-21T02:06:16.689517+0000","last_fullsized":"2024-09-21T02:06:16.689517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'215","reported_seq":233,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936059+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:07:15.936059+0000","last_peered":"2024-09-21T02:07:15.936059+0000","last_clean":"2024-09-21T02:07:15.936059+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:07:15.936059+0000","last_undegraded":"2024-09-21T02:07:15.936059+0000","last_fullsized":"2024-09-21T02:07:15.936059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:11.588175+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:06:11.588175+0000","last_peered":"2024-09-21T02:06:11.588175+0000","last_clean":"2024-09-21T02:06:11.588175+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:06:11.588175+0000","last_undegraded":"2024-09-21T02:06:11.588175+0000","last_fullsized":"2024-09-21T02:06:11.588175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.933136+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:07:15.933136+0000","last_peered":"2024-09-21T02:07:15.933136+0000","last_clean":"2024-09-21T02:07:15.933136+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:07:15.933136+0000","last_undegraded":"2024-09-21T02:07:15.933136+0000","last_fullsized":"2024-09-21T02:07:15.933136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208099+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:45.208099+0000","last_peered":"2024-09-21T02:04:45.208099+0000","last_clean":"2024-09-21T02:04:45.208099+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:45.208099+0000","last_undegraded":"2024-09-21T02:04:45.208099+0000","last_fullsized":"2024-09-21T02:04:45.208099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936043+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:07:15.936043+0000","last_peered":"2024-09-21T02:07:15.936043+0000","last_clean":"2024-09-21T02:07:15.936043+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:07:15.936043+0000","last_undegraded":"2024-09-21T02:07:15.936043+0000","last_fullsized":"2024-09-21T02:07:15.936043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208098+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:04:45.208098+0000","last_peered":"2024-09-21T02:04:45.208098+0000","last_clean":"2024-09-21T02:04:45.208098+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:04:45.208098+0000","last_undegraded":"2024-09-21T02:04:45.208098+0000","last_fullsized":"2024-09-21T02:04:45.208098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'165","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932552+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:07:15.932552+0000","last_peered":"2024-09-21T02:07:15.932552+0000","last_clean":"2024-09-21T02:07:15.932552+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:07:15.932552+0000","last_undegraded":"2024-09-21T02:07:15.932552+0000","last_fullsized":"2024-09-21T02:07:15.932552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207953+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:04:45.207953+0000","last_peered":"2024-09-21T02:04:45.207953+0000","last_clean":"2024-09-21T02:04:45.207953+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:04:45.207953+0000","last_undegraded":"2024-09-21T02:04:45.207953+0000","last_fullsized":"2024-09-21T02:04:45.207953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936586+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:07:15.936586+0000","last_peered":"2024-09-21T02:07:15.936586+0000","last_clean":"2024-09-21T02:07:15.936586+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:07:15.936586+0000","last_undegraded":"2024-09-21T02:07:15.936586+0000","last_fullsized":"2024-09-21T02:07:15.936586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211129+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:04:45.211129+0000","last_peered":"2024-09-21T02:04:45.211129+0000","last_clean":"2024-09-21T02:04:45.211129+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:04:45.211129+0000","last_undegraded":"2024-09-21T02:04:45.211129+0000","last_fullsized":"2024-09-21T02:04:45.211129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205332+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:04:45.205332+0000","last_peered":"2024-09-21T02:04:45.205332+0000","last_clean":"2024-09-21T02:04:45.205332+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:04:45.205332+0000","last_undegraded":"2024-09-21T02:04:45.205332+0000","last_fullsized":"2024-09-21T02:04:45.205332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.211254+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:06:44.211254+0000","last_peered":"2024-09-21T02:06:44.211254+0000","last_clean":"2024-09-21T02:06:44.211254+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:06:44.211254+0000","last_undegraded":"2024-09-21T02:06:44.211254+0000","last_fullsized":"2024-09-21T02:06:44.211254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210923+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:04:45.210923+0000","last_peered":"2024-09-21T02:04:45.210923+0000","last_clean":"2024-09-21T02:04:45.210923+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:04:45.210923+0000","last_undegraded":"2024-09-21T02:04:45.210923+0000","last_fullsized":"2024-09-21T02:04:45.210923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.562728+0000","last_change":"2024-09-21T02:03:58.080691+0000","last_active":"2024-09-21T02:06:44.562728+0000","last_peered":"2024-09-21T02:06:44.562728+0000","last_clean":"2024-09-21T02:06:44.562728+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:06:44.562728+0000","last_undegraded":"2024-09-21T02:06:44.562728+0000","last_fullsized":"2024-09-21T02:06:44.562728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:05.466556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:47.179470+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:06:47.179470+0000","last_peered":"2024-09-21T02:06:47.179470+0000","last_clean":"2024-09-21T02:06:47.179470+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:06:47.179470+0000","last_undegraded":"2024-09-21T02:06:47.179470+0000","last_fullsized":"2024-09-21T02:06:47.179470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":595,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.494702+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:06:44.494702+0000","last_peered":"2024-09-21T02:06:44.494702+0000","last_clean":"2024-09-21T02:06:44.494702+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:06:44.494702+0000","last_undegraded":"2024-09-21T02:06:44.494702+0000","last_fullsized":"2024-09-21T02:06:44.494702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205672+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:04:45.205672+0000","last_peered":"2024-09-21T02:04:45.205672+0000","last_clean":"2024-09-21T02:04:45.205672+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:04:45.205672+0000","last_undegraded":"2024-09-21T02:04:45.205672+0000","last_fullsized":"2024-09-21T02:04:45.205672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.923138+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:07:15.923138+0000","last_peered":"2024-09-21T02:07:15.923138+0000","last_clean":"2024-09-21T02:07:15.923138+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:07:15.923138+0000","last_undegraded":"2024-09-21T02:07:15.923138+0000","last_fullsized":"2024-09-21T02:07:15.923138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207553+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:04:45.207553+0000","last_peered":"2024-09-21T02:04:45.207553+0000","last_clean":"2024-09-21T02:04:45.207553+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:04:45.207553+0000","last_undegraded":"2024-09-21T02:04:45.207553+0000","last_fullsized":"2024-09-21T02:04:45.207553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934011+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:07:15.934011+0000","last_peered":"2024-09-21T02:07:15.934011+0000","last_clean":"2024-09-21T02:07:15.934011+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:07:15.934011+0000","last_undegraded":"2024-09-21T02:07:15.934011+0000","last_fullsized":"2024-09-21T02:07:15.934011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211858+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:04:45.211858+0000","last_peered":"2024-09-21T02:04:45.211858+0000","last_clean":"2024-09-21T02:04:45.211858+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:04:45.211858+0000","last_undegraded":"2024-09-21T02:04:45.211858+0000","last_fullsized":"2024-09-21T02:04:45.211858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.555740+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:06:44.555740+0000","last_peered":"2024-09-21T02:06:44.555740+0000","last_clean":"2024-09-21T02:06:44.555740+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:06:44.555740+0000","last_undegraded":"2024-09-21T02:06:44.555740+0000","last_fullsized":"2024-09-21T02:06:44.555740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'21","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:43.130438+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:06:43.130438+0000","last_peered":"2024-09-21T02:06:43.130438+0000","last_clean":"2024-09-21T02:06:43.130438+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:06:43.130438+0000","last_undegraded":"2024-09-21T02:06:43.130438+0000","last_fullsized":"2024-09-21T02:06:43.130438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'161","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.495833+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:06:44.495833+0000","last_peered":"2024-09-21T02:06:44.495833+0000","last_clean":"2024-09-21T02:06:44.495833+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:06:44.495833+0000","last_undegraded":"2024-09-21T02:06:44.495833+0000","last_fullsized":"2024-09-21T02:06:44.495833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:09.584145+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:06:09.584145+0000","last_peered":"2024-09-21T02:06:09.584145+0000","last_clean":"2024-09-21T02:06:09.584145+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:06:09.584145+0000","last_undegraded":"2024-09-21T02:06:09.584145+0000","last_fullsized":"2024-09-21T02:06:09.584145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.520907+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:06:44.520907+0000","last_peered":"2024-09-21T02:06:44.520907+0000","last_clean":"2024-09-21T02:06:44.520907+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:06:44.520907+0000","last_undegraded":"2024-09-21T02:06:44.520907+0000","last_fullsized":"2024-09-21T02:06:44.520907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210942+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:04:45.210942+0000","last_peered":"2024-09-21T02:04:45.210942+0000","last_clean":"2024-09-21T02:04:45.210942+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:04:45.210942+0000","last_undegraded":"2024-09-21T02:04:45.210942+0000","last_fullsized":"2024-09-21T02:04:45.210942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.909177+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:07:15.909177+0000","last_peered":"2024-09-21T02:07:15.909177+0000","last_clean":"2024-09-21T02:07:15.909177+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:07:15.909177+0000","last_undegraded":"2024-09-21T02:07:15.909177+0000","last_fullsized":"2024-09-21T02:07:15.909177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207751+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:04:45.207751+0000","last_peered":"2024-09-21T02:04:45.207751+0000","last_clean":"2024-09-21T02:04:45.207751+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:04:45.207751+0000","last_undegraded":"2024-09-21T02:04:45.207751+0000","last_fullsized":"2024-09-21T02:04:45.207751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.214359+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:06:44.214359+0000","last_peered":"2024-09-21T02:06:44.214359+0000","last_clean":"2024-09-21T02:06:44.214359+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:06:44.214359+0000","last_undegraded":"2024-09-21T02:06:44.214359+0000","last_fullsized":"2024-09-21T02:06:44.214359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207926+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:04:45.207926+0000","last_peered":"2024-09-21T02:04:45.207926+0000","last_clean":"2024-09-21T02:04:45.207926+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:04:45.207926+0000","last_undegraded":"2024-09-21T02:04:45.207926+0000","last_fullsized":"2024-09-21T02:04:45.207926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'156","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.499249+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:06:44.499249+0000","last_peered":"2024-09-21T02:06:44.499249+0000","last_clean":"2024-09-21T02:06:44.499249+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:06:44.499249+0000","last_undegraded":"2024-09-21T02:06:44.499249+0000","last_fullsized":"2024-09-21T02:06:44.499249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.141927+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:05:47.141927+0000","last_peered":"2024-09-21T02:05:47.141927+0000","last_clean":"2024-09-21T02:05:47.141927+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:05:47.141927+0000","last_undegraded":"2024-09-21T02:05:47.141927+0000","last_fullsized":"2024-09-21T02:05:47.141927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.537214+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:06:44.537214+0000","last_peered":"2024-09-21T02:06:44.537214+0000","last_clean":"2024-09-21T02:06:44.537214+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:06:44.537214+0000","last_undegraded":"2024-09-21T02:06:44.537214+0000","last_fullsized":"2024-09-21T02:06:44.537214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'205","reported_seq":223,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.930285+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:07:15.930285+0000","last_peered":"2024-09-21T02:07:15.930285+0000","last_clean":"2024-09-21T02:07:15.930285+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:07:15.930285+0000","last_undegraded":"2024-09-21T02:07:15.930285+0000","last_fullsized":"2024-09-21T02:07:15.930285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211388+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:45.211388+0000","last_peered":"2024-09-21T02:04:45.211388+0000","last_clean":"2024-09-21T02:04:45.211388+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:45.211388+0000","last_undegraded":"2024-09-21T02:04:45.211388+0000","last_fullsized":"2024-09-21T02:04:45.211388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934153+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:07:15.934153+0000","last_peered":"2024-09-21T02:07:15.934153+0000","last_clean":"2024-09-21T02:07:15.934153+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:07:15.934153+0000","last_undegraded":"2024-09-21T02:07:15.934153+0000","last_fullsized":"2024-09-21T02:07:15.934153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:08.902449+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:06:08.902449+0000","last_peered":"2024-09-21T02:06:08.902449+0000","last_clean":"2024-09-21T02:06:08.902449+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:06:08.902449+0000","last_undegraded":"2024-09-21T02:06:08.902449+0000","last_fullsized":"2024-09-21T02:06:08.902449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.244570+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:06:44.244570+0000","last_peered":"2024-09-21T02:06:44.244570+0000","last_clean":"2024-09-21T02:06:44.244570+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:06:44.244570+0000","last_undegraded":"2024-09-21T02:06:44.244570+0000","last_fullsized":"2024-09-21T02:06:44.244570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280872+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:04:45.280872+0000","last_peered":"2024-09-21T02:04:45.280872+0000","last_clean":"2024-09-21T02:04:45.280872+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:04:45.280872+0000","last_undegraded":"2024-09-21T02:04:45.280872+0000","last_fullsized":"2024-09-21T02:04:45.280872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'176","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:44.490694+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:06:44.490694+0000","last_peered":"2024-09-21T02:06:44.490694+0000","last_clean":"2024-09-21T02:06:44.490694+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:06:44.490694+0000","last_undegraded":"2024-09-21T02:06:44.490694+0000","last_fullsized":"2024-09-21T02:06:44.490694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280763+0000","last_change":"2024-09-21T02:03:57.069618+0000","last_active":"2024-09-21T02:04:45.280763+0000","last_peered":"2024-09-21T02:04:45.280763+0000","last_clean":"2024-09-21T02:04:45.280763+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:04:45.280763+0000","last_undegraded":"2024-09-21T02:04:45.280763+0000","last_fullsized":"2024-09-21T02:04:45.280763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:14.862543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10984,"ondisk_log_size":10984,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7623565,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3820,"num_write_kb":11014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15405056,"data_stored":15247444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3795,"ondisk_log_size":3795,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57564,"kb_used_data":4172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314276,"statfs":{"total":96636764160,"available":96577818624,"internally_reserved":0,"allocated":4272128,"data_stored":4100839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":86315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":86048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83228,"kb_used_data":7692,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94288612,"statfs":{"total":96636764160,"available":96551538688,"internally_reserved":0,"allocated":7876608,"data_stored":7705348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":86260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":85881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53572,"kb_used_data":4340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318268,"statfs":{"total":96636764160,"available":96581906432,"internally_reserved":0,"allocated":4444160,"data_stored":4280847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":843776,"data_stored":675965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7643136,"data_stored":7620411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3448832,"data_stored":3425707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:07:17.219 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:07:17.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.368+0000 7fc99b484640 1 -- 172.21.15.88:0/2195802245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 msgr2=0x7fc994153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.368+0000 7fc99b484640 1 --2- 172.21.15.88:0/2195802245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc994153730 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc984009830 tx=0x7fc98402a510 comp rx=0 tx=0).stop 2024-09-21T02:07:17.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.368+0000 7fc99b484640 1 -- 172.21.15.88:0/2195802245 shutdown_connections 2024-09-21T02:07:17.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.368+0000 7fc99b484640 1 --2- 172.21.15.88:0/2195802245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc994153730 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.368+0000 7fc99b484640 1 -- 172.21.15.88:0/2195802245 >> 172.21.15.88:0/2195802245 conn(0x7fc994073210 msgr2=0x7fc9940cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:17.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.368+0000 7fc99b484640 1 -- 172.21.15.88:0/2195802245 shutdown_connections 2024-09-21T02:07:17.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.368+0000 7fc99b484640 1 -- 172.21.15.88:0/2195802245 wait complete. 2024-09-21T02:07:17.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.370+0000 7fc99b484640 1 Processor -- start 2024-09-21T02:07:17.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.370+0000 7fc99b484640 1 -- start start 2024-09-21T02:07:17.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc99b484640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9940c41a0 0x7fc9940c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc99b484640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9940c99a0 0x7fc9941e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc99b484640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc9940ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc99b484640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc994070dc0 con 0x7fc994153320 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc99b484640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc994070ae0 con 0x7fc9940c41a0 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc99b484640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940c4cc0 con 0x7fc9940c99a0 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc999a21640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc9940ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc999a21640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc9940ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39602/0 (socket says 172.21.15.88:39602) 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc999a21640 1 -- 172.21.15.88:0/3217239198 learned_addr learned my addr 172.21.15.88:0/3217239198 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc999220640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9940c41a0 0x7fc9940c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.371+0000 7fc998a1f640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9940c99a0 0x7fc9941e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc999a21640 1 -- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9940c99a0 msgr2=0x7fc9941e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc999a21640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9940c99a0 0x7fc9941e4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc999a21640 1 -- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9940c41a0 msgr2=0x7fc9940c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc999a21640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9940c41a0 0x7fc9940c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc999a21640 1 -- 172.21.15.88:0/3217239198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99406f130 con 0x7fc994153320 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc999220640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9940c41a0 0x7fc9940c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:17.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc999a21640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc9940ce080 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00cab0 tx=0x7fc98c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:17.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc989ffb640 1 -- 172.21.15.88:0/3217239198 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c016020 con 0x7fc994153320 2024-09-21T02:07:17.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc989ffb640 1 -- 172.21.15.88:0/3217239198 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc98c00ad20 con 0x7fc994153320 2024-09-21T02:07:17.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99406f3d0 con 0x7fc994153320 2024-09-21T02:07:17.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc989ffb640 1 -- 172.21.15.88:0/3217239198 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c012660 con 0x7fc994153320 2024-09-21T02:07:17.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.372+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99406fc80 con 0x7fc994153320 2024-09-21T02:07:17.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.373+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9940c5370 con 0x7fc994153320 2024-09-21T02:07:17.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.374+0000 7fc989ffb640 1 -- 172.21.15.88:0/3217239198 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc98c019020 con 0x7fc994153320 2024-09-21T02:07:17.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.374+0000 7fc989ffb640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc978078b90 0x7fc97807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:17.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.374+0000 7fc989ffb640 1 -- 172.21.15.88:0/3217239198 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc98c09b080 con 0x7fc994153320 2024-09-21T02:07:17.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.374+0000 7fc999220640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc978078b90 0x7fc97807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:17.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.375+0000 7fc999220640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc978078b90 0x7fc97807b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc990006fd0 tx=0x7fc990008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:17.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.379+0000 7fc989ffb640 1 -- 172.21.15.88:0/3217239198 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc9940c5370 con 0x7fc994153320 2024-09-21T02:07:17.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.558+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc994159330 con 0x7fc994153320 2024-09-21T02:07:17.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.559+0000 7fc989ffb640 1 -- 172.21.15.88:0/3217239198 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16915 (secure 0 0 0) 0x7fc994159330 con 0x7fc994153320 2024-09-21T02:07:17.559 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:17.560 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:04:45.193137+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc978078b90 msgr2=0x7fc97807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc978078b90 0x7fc97807b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc990006fd0 tx=0x7fc990008040 comp rx=0 tx=0).stop 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 msgr2=0x7fc9940ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc9940ce080 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00cab0 tx=0x7fc98c00cf70 comp rx=0 tx=0).stop 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 shutdown_connections 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc978078b90 0x7fc97807b0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc994153320 0x7fc9940ce080 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9940c99a0 0x7fc9941e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 --2- 172.21.15.88:0/3217239198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9940c41a0 0x7fc9940c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 >> 172.21.15.88:0/3217239198 conn(0x7fc994073210 msgr2=0x7fc9941e5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 shutdown_connections 2024-09-21T02:07:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:17.561+0000 7fc99b484640 1 -- 172.21.15.88:0/3217239198 wait complete. 2024-09-21T02:07:18.574 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T02:07:18.574 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 -- 172.21.15.88:0/2192874366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 msgr2=0x7fbe28153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 --2- 172.21.15.88:0/2192874366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 0x7fbe28153720 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbe280c4880 tx=0x7fbe1802a580 comp rx=0 tx=0).stop 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 -- 172.21.15.88:0/2192874366 shutdown_connections 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 --2- 172.21.15.88:0/2192874366 >> v2:172.21.15.195:3300/0 conn(0x7fbe280c9b10 0x7fbe280c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 --2- 172.21.15.88:0/2192874366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe280cbba0 0x7fbe280cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 --2- 172.21.15.88:0/2192874366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 0x7fbe28153720 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 -- 172.21.15.88:0/2192874366 >> 172.21.15.88:0/2192874366 conn(0x7fbe28068d00 msgr2=0x7fbe28069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 -- 172.21.15.88:0/2192874366 shutdown_connections 2024-09-21T02:07:18.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.696+0000 7fbe2d119640 1 -- 172.21.15.88:0/2192874366 wait complete. 2024-09-21T02:07:18.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.698+0000 7fbe2d119640 1 Processor -- start 2024-09-21T02:07:18.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.698+0000 7fbe2d119640 1 -- start start 2024-09-21T02:07:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe2d119640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 0x7fbe281e0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe2d119640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe280c9b10 0x7fbe28151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe2d119640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe280cbba0 0x7fbe28154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe2d119640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28159a80 con 0x7fbe280c4130 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe2d119640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe280cb620 con 0x7fbe280c9b10 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe2d119640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fbe28159c00 con 0x7fbe280cbba0 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe26d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 0x7fbe281e0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe26d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 0x7fbe281e0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49088/0 (socket says 172.21.15.88:49088) 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.699+0000 7fbe26d9d640 1 -- 172.21.15.88:0/3409867424 learned_addr learned my addr 172.21.15.88:0/3409867424 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe2759e640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe280cbba0 0x7fbe28154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe1ffff640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe280c9b10 0x7fbe28151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe26d9d640 1 -- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe280cbba0 msgr2=0x7fbe28154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe26d9d640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe280cbba0 0x7fbe28154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe26d9d640 1 -- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe280c9b10 msgr2=0x7fbe28151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe26d9d640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe280c9b10 0x7fbe28151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe26d9d640 1 -- 172.21.15.88:0/3409867424 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe281e0cb0 con 0x7fbe280c4130 2024-09-21T02:07:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe26d9d640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 0x7fbe281e0770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034040 tx=0x7fbe180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1803e030 con 0x7fbe280c4130 2024-09-21T02:07:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe18003d90 con 0x7fbe280c4130 2024-09-21T02:07:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18035040 con 0x7fbe280c4130 2024-09-21T02:07:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe280b35d0 con 0x7fbe280c4130 2024-09-21T02:07:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.700+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281e0220 con 0x7fbe280c4130 2024-09-21T02:07:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.701+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe28153720 con 0x7fbe280c4130 2024-09-21T02:07:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.702+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe18041030 con 0x7fbe280c4130 2024-09-21T02:07:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.702+0000 7fbe1f7fe640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbe08078b90 0x7fbe0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.702+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbe180bbac0 con 0x7fbe280c4130 2024-09-21T02:07:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.703+0000 7fbe1ffff640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbe08078b90 0x7fbe0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.703+0000 7fbe1f7fe640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbe08080f60 0x7fbe08083430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.703+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 --> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe18005310 con 0x7fbe08080f60 2024-09-21T02:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.703+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbe180bbf50 con 0x7fbe280c4130 2024-09-21T02:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.703+0000 7fbe2759e640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbe08080f60 0x7fbe08083430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.703+0000 7fbe1ffff640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbe08078b90 0x7fbe0807b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe280c9530 tx=0x7fbe1000b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.703+0000 7fbe2759e640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbe08080f60 0x7fbe08083430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.704+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== osd.0 v2:172.21.15.88:6800/1826444869 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe18005310 con 0x7fbe08080f60 2024-09-21T02:07:18.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.716+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 --> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbe280b2360 con 0x7fbe08080f60 2024-09-21T02:07:18.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.716+0000 7fbe1f7fe640 1 -- 172.21.15.88:0/3409867424 <== osd.0 v2:172.21.15.88:6800/1826444869 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbe280b2360 con 0x7fbe08080f60 2024-09-21T02:07:18.717 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:18.717 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:18.717 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:18.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbe08080f60 msgr2=0x7fbe08083430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:6800/1826444869,v1:172.21.15.88:6801/1826444869] conn(0x7fbe08080f60 0x7fbe08083430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbe08078b90 msgr2=0x7fbe0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fbe08078b90 0x7fbe0807b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe280c9530 tx=0x7fbe1000b260 comp rx=0 tx=0).stop 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 msgr2=0x7fbe281e0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 --2- 172.21.15.88:0/3409867424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe280c4130 0x7fbe281e0770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034040 tx=0x7fbe180038a0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2759e640 1 -- 172.21.15.88:0/3409867424 reap_dead start 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 shutdown_connections 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 >> 172.21.15.88:0/3409867424 conn(0x7fbe28068d00 msgr2=0x7fbe281575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 shutdown_connections 2024-09-21T02:07:18.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.717+0000 7fbe2d119640 1 -- 172.21.15.88:0/3409867424 wait complete. 2024-09-21T02:07:18.728 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T02:07:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.841+0000 7fa211c5c640 1 -- 172.21.15.88:0/1220379257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4003a50 msgr2=0x7fa1f4003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.841+0000 7fa211c5c640 1 --2- 172.21.15.88:0/1220379257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4003a50 0x7fa1f4003ee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0016c0 tx=0x7fa1f4005830 comp rx=0 tx=0).stop 2024-09-21T02:07:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.841+0000 7fa211c5c640 1 -- 172.21.15.88:0/1220379257 shutdown_connections 2024-09-21T02:07:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.841+0000 7fa211c5c640 1 --2- 172.21.15.88:0/1220379257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4003a50 0x7fa1f4003ee0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.841+0000 7fa211c5c640 1 -- 172.21.15.88:0/1220379257 >> 172.21.15.88:0/1220379257 conn(0x7fa20c06de70 msgr2=0x7fa20c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.841+0000 7fa211c5c640 1 -- 172.21.15.88:0/1220379257 shutdown_connections 2024-09-21T02:07:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.841+0000 7fa211c5c640 1 -- 172.21.15.88:0/1220379257 wait complete. 2024-09-21T02:07:18.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.843+0000 7fa211c5c640 1 Processor -- start 2024-09-21T02:07:18.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.843+0000 7fa211c5c640 1 -- start start 2024-09-21T02:07:18.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.844+0000 7fa211c5c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1f4003a50 0x7fa20c14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.844+0000 7fa211c5c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa20c1f5600 0x7fa20c157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.844+0000 7fa211c5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa20c157b90 0x7fa20c06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.844+0000 7fa211c5c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c0c9c50 con 0x7fa20c157b90 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.844+0000 7fa211c5c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c0c9970 con 0x7fa20c1f5600 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.844+0000 7fa211c5c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c072190 con 0x7fa1f4003a50 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa20c157b90 0x7fa20c06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa20c157b90 0x7fa20c06cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49114/0 (socket says 172.21.15.88:49114) 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 -- 172.21.15.88:0/3192376322 learned_addr learned my addr 172.21.15.88:0/3192376322 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20b7fe640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1f4003a50 0x7fa20c14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20affd640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa20c1f5600 0x7fa20c157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 -- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1f4003a50 msgr2=0x7fa20c14c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1f4003a50 0x7fa20c14c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 -- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa20c1f5600 msgr2=0x7fa20c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa20c1f5600 0x7fa20c157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 -- 172.21.15.88:0/3192376322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c1544c0 con 0x7fa20c157b90 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa20bfff640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa20c157b90 0x7fa20c06cd60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa204005fd0 tx=0x7fa20400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa1f3fff640 1 -- 172.21.15.88:0/3192376322 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204017070 con 0x7fa20c157b90 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa1f3fff640 1 -- 172.21.15.88:0/3192376322 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa204015070 con 0x7fa20c157b90 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c1547a0 con 0x7fa20c157b90 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa1f3fff640 1 -- 172.21.15.88:0/3192376322 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204003df0 con 0x7fa20c157b90 2024-09-21T02:07:18.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.845+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c14e830 con 0x7fa20c157b90 2024-09-21T02:07:18.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.846+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20c1549a0 con 0x7fa20c157b90 2024-09-21T02:07:18.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.847+0000 7fa1f3fff640 1 -- 172.21.15.88:0/3192376322 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa204004030 con 0x7fa20c157b90 2024-09-21T02:07:18.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.847+0000 7fa1f3fff640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1f8078b90 0x7fa1f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:18.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.847+0000 7fa1f3fff640 1 -- 172.21.15.88:0/3192376322 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa20409b600 con 0x7fa20c157b90 2024-09-21T02:07:18.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.848+0000 7fa20b7fe640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1f8078b90 0x7fa1f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:18.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.848+0000 7fa20b7fe640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1f8078b90 0x7fa1f807b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc032000 tx=0x7fa1fc02c040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:18.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:18.852+0000 7fa1f3fff640 1 -- 172.21.15.88:0/3192376322 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa20c1549a0 con 0x7fa20c157b90 2024-09-21T02:07:19.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.029+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa20c0c3670 con 0x7fa1f8078b90 2024-09-21T02:07:19.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.029+0000 7fa1f3fff640 1 -- 172.21.15.88:0/3192376322 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa20c0c3670 con 0x7fa1f8078b90 2024-09-21T02:07:19.029 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T02:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1f8078b90 msgr2=0x7fa1f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1f8078b90 0x7fa1f807b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc032000 tx=0x7fa1fc02c040 comp rx=0 tx=0).stop 2024-09-21T02:07:19.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa20c157b90 msgr2=0x7fa20c06cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa20c157b90 0x7fa20c06cd60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa204005fd0 tx=0x7fa20400e720 comp rx=0 tx=0).stop 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 shutdown_connections 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa1f8078b90 0x7fa1f807b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa20c157b90 0x7fa20c06cd60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa20c1f5600 0x7fa20c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 --2- 172.21.15.88:0/3192376322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1f4003a50 0x7fa20c14c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 >> 172.21.15.88:0/3192376322 conn(0x7fa20c06de70 msgr2=0x7fa20c0cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 shutdown_connections 2024-09-21T02:07:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.032+0000 7fa211c5c640 1 -- 172.21.15.88:0/3192376322 wait complete. 2024-09-21T02:07:19.044 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T02:07:19.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:19.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 -- 172.21.15.88:0/2335261056 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93740c9b10 msgr2=0x7f93740c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 --2- 172.21.15.88:0/2335261056 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93740c9b10 0x7f93740c9f20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9364005fd0 tx=0x7f936402a5d0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 -- 172.21.15.88:0/2335261056 shutdown_connections 2024-09-21T02:07:19.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 --2- 172.21.15.88:0/2335261056 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93740c9b10 0x7f93740c9f20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 --2- 172.21.15.88:0/2335261056 >> v2:172.21.15.195:3300/0 conn(0x7f93740c4130 0x7f9374153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 -- 172.21.15.88:0/2335261056 >> 172.21.15.88:0/2335261056 conn(0x7f9374068d00 msgr2=0x7f9374069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 -- 172.21.15.88:0/2335261056 shutdown_connections 2024-09-21T02:07:19.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.161+0000 7f937bf12640 1 -- 172.21.15.88:0/2335261056 wait complete. 2024-09-21T02:07:19.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.163+0000 7f937bf12640 1 Processor -- start 2024-09-21T02:07:19.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.163+0000 7f937bf12640 1 -- start start 2024-09-21T02:07:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937bf12640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93740c4130 0x7f93741df810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937bf12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93740c9b10 0x7f93741e3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937bf12640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93740cbba0 0x7f93741e1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937bf12640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f937415a190 con 0x7f93740cbba0 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937bf12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f93740cb620 con 0x7f93740c9b10 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937bf12640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f937415a310 con 0x7f93740c4130 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937a4af640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93740cbba0 0x7f93741e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937a4af640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93740cbba0 0x7f93741e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49124/0 (socket says 172.21.15.88:49124) 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f937a4af640 1 -- 172.21.15.88:0/632033857 learned_addr learned my addr 172.21.15.88:0/632033857 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f93794ad640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93740c9b10 0x7f93741e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.164+0000 7f9379cae640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93740c4130 0x7f93741df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937a4af640 1 -- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93740c4130 msgr2=0x7f93741df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937a4af640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93740c4130 0x7f93741df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937a4af640 1 -- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93740c9b10 msgr2=0x7f93741e3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937a4af640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93740c9b10 0x7f93741e3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937a4af640 1 -- 172.21.15.88:0/632033857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93740b1860 con 0x7f93740cbba0 2024-09-21T02:07:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937a4af640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93740cbba0 0x7f93741e1630 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f936c00c700 tx=0x7f936c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c003b70 con 0x7f93740cbba0 2024-09-21T02:07:19.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f936c003d10 con 0x7f93740cbba0 2024-09-21T02:07:19.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93740b1b40 con 0x7f93740cbba0 2024-09-21T02:07:19.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c00aab0 con 0x7f93740cbba0 2024-09-21T02:07:19.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.165+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93740cfc80 con 0x7f93740cbba0 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.166+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93741e3a80 con 0x7f93740cbba0 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.167+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f936c00ac50 con 0x7f93740cbba0 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.167+0000 7f936a7fc640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9358078b90 0x7f935807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f936c09b4f0 con 0x7f93740cbba0 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f936a7fc640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7f9358080f60 0x7f9358083430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f9379cae640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9358078b90 0x7f935807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 --> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f936c019e80 con 0x7f9358080f60 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f936c09b980 con 0x7f93740cbba0 2024-09-21T02:07:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f93794ad640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7f9358080f60 0x7f9358083430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f93794ad640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7f9358080f60 0x7f9358083430 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.168+0000 7f9379cae640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9358078b90 0x7f935807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9370009b30 tx=0x7f9370005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.169+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== osd.1 v2:172.21.15.88:6824/2879208543 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f936c019e80 con 0x7f9358080f60 2024-09-21T02:07:19.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.181+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 --> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f93740cf7c0 con 0x7f9358080f60 2024-09-21T02:07:19.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.181+0000 7f936a7fc640 1 -- 172.21.15.88:0/632033857 <== osd.1 v2:172.21.15.88:6824/2879208543 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f93740cf7c0 con 0x7f9358080f60 2024-09-21T02:07:19.181 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:19.182 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:19.182 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:19.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7f9358080f60 msgr2=0x7f9358083430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:6824/2879208543,v1:172.21.15.88:6825/2879208543] conn(0x7f9358080f60 0x7f9358083430 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9358078b90 msgr2=0x7f935807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9358078b90 0x7f935807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9370009b30 tx=0x7f9370005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93740cbba0 msgr2=0x7f93741e1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 --2- 172.21.15.88:0/632033857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93740cbba0 0x7f93741e1630 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f936c00c700 tx=0x7f936c00cbc0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937a4af640 1 -- 172.21.15.88:0/632033857 reap_dead start 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 shutdown_connections 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 >> 172.21.15.88:0/632033857 conn(0x7f9374068d00 msgr2=0x7f9374152c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 shutdown_connections 2024-09-21T02:07:19.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.182+0000 7f937bf12640 1 -- 172.21.15.88:0/632033857 wait complete. 2024-09-21T02:07:19.192 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T02:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.306+0000 7fd9ee2be640 1 -- 172.21.15.88:0/1614054492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e81630c0 msgr2=0x7fd9e81654e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.306+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/1614054492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e81630c0 0x7fd9e81654e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0005fd0 tx=0x7fd9e002a8b0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.306+0000 7fd9ee2be640 1 -- 172.21.15.88:0/1614054492 shutdown_connections 2024-09-21T02:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.306+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/1614054492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e81630c0 0x7fd9e81654e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.306+0000 7fd9ee2be640 1 -- 172.21.15.88:0/1614054492 >> 172.21.15.88:0/1614054492 conn(0x7fd9e80720b0 msgr2=0x7fd9e80cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.307+0000 7fd9ee2be640 1 -- 172.21.15.88:0/1614054492 shutdown_connections 2024-09-21T02:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.307+0000 7fd9ee2be640 1 -- 172.21.15.88:0/1614054492 wait complete. 2024-09-21T02:07:19.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.308+0000 7fd9ee2be640 1 Processor -- start 2024-09-21T02:07:19.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.308+0000 7fd9ee2be640 1 -- start start 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ee2be640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e80c97d0 0x7fd9e81f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ee2be640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e8156d60 0x7fd9e806e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ee2be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e81630c0 0x7fd9e81522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ee2be640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e80c9d00 con 0x7fd9e80c97d0 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ee2be640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e80c4830 con 0x7fd9e8156d60 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ee2be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e80758e0 con 0x7fd9e81630c0 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e80c97d0 0x7fd9e81f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e80c97d0 0x7fd9e81f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49132/0 (socket says 172.21.15.88:49132) 2024-09-21T02:07:19.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 -- 172.21.15.88:0/3581042330 learned_addr learned my addr 172.21.15.88:0/3581042330 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:19.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e77fe640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e8156d60 0x7fd9e806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ec85b640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e81630c0 0x7fd9e81522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 -- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e81630c0 msgr2=0x7fd9e81522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e81630c0 0x7fd9e81522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 -- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e8156d60 msgr2=0x7fd9e806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e8156d60 0x7fd9e806e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 -- 172.21.15.88:0/3581042330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e8154610 con 0x7fd9e80c97d0 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e77fe640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e8156d60 0x7fd9e806e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ec85b640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e81630c0 0x7fd9e81522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e7fff640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e80c97d0 0x7fd9e81f3390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c430 tx=0x7fd9d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e4ff9640 1 -- 172.21.15.88:0/3581042330 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011790 con 0x7fd9e80c97d0 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e81f38d0 con 0x7fd9e80c97d0 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.310+0000 7fd9e4ff9640 1 -- 172.21.15.88:0/3581042330 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9d8017070 con 0x7fd9e80c97d0 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.311+0000 7fd9e4ff9640 1 -- 172.21.15.88:0/3581042330 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8011ce0 con 0x7fd9e80c97d0 2024-09-21T02:07:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.311+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e81560d0 con 0x7fd9e80c97d0 2024-09-21T02:07:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.312+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e80c3550 con 0x7fd9e80c97d0 2024-09-21T02:07:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.312+0000 7fd9e4ff9640 1 -- 172.21.15.88:0/3581042330 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9d801c460 con 0x7fd9e80c97d0 2024-09-21T02:07:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.313+0000 7fd9e4ff9640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.313+0000 7fd9e4ff9640 1 -- 172.21.15.88:0/3581042330 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd9d809ab80 con 0x7fd9e80c97d0 2024-09-21T02:07:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.313+0000 7fd9e77fe640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.314+0000 7fd9e77fe640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80c89b0 tx=0x7fd9dc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.317+0000 7fd9e4ff9640 1 -- 172.21.15.88:0/3581042330 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd9d8062fe0 con 0x7fd9e80c97d0 2024-09-21T02:07:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.495+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd9e814afb0 con 0x7fd9d4078bc0 2024-09-21T02:07:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.496+0000 7fd9e4ff9640 1 -- 172.21.15.88:0/3581042330 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd9e814afb0 con 0x7fd9d4078bc0 2024-09-21T02:07:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T02:07:19.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd9d4078bc0 msgr2=0x7fd9d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80c89b0 tx=0x7fd9dc009040 comp rx=0 tx=0).stop 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e80c97d0 msgr2=0x7fd9e81f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e80c97d0 0x7fd9e81f3390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c430 tx=0x7fd9d800c8f0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 shutdown_connections 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd9e81630c0 0x7fd9e81522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9e8156d60 0x7fd9e806e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 --2- 172.21.15.88:0/3581042330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9e80c97d0 0x7fd9e81f3390 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 >> 172.21.15.88:0/3581042330 conn(0x7fd9e80720b0 msgr2=0x7fd9e80c9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 shutdown_connections 2024-09-21T02:07:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.498+0000 7fd9ee2be640 1 -- 172.21.15.88:0/3581042330 wait complete. 2024-09-21T02:07:19.510 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T02:07:19.510 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:19.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/2890659670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d08150220 msgr2=0x7f7d080cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 --2- 172.21.15.88:0/2890659670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d08150220 0x7f7d080cacf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7d0815a040 tx=0x7f7cf802a810 comp rx=0 tx=0).stop 2024-09-21T02:07:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/2890659670 shutdown_connections 2024-09-21T02:07:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 --2- 172.21.15.88:0/2890659670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d080c49f0 0x7f7d080c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 --2- 172.21.15.88:0/2890659670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d08150220 0x7f7d080cacf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/2890659670 >> 172.21.15.88:0/2890659670 conn(0x7f7d080690f0 msgr2=0x7f7d08069500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/2890659670 shutdown_connections 2024-09-21T02:07:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.626+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/2890659670 wait complete. 2024-09-21T02:07:19.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.628+0000 7f7d0cc9b640 1 Processor -- start 2024-09-21T02:07:19.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.628+0000 7f7d0cc9b640 1 -- start start 2024-09-21T02:07:19.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0cc9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080c4030 0x7f7d081f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0cc9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d080c49f0 0x7f7d081562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0cc9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d08150220 0x7f7d081f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0cc9b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08159b50 con 0x7f7d080c4030 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0cc9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08159870 con 0x7f7d080c49f0 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0cc9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080724a0 con 0x7f7d08150220 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080c4030 0x7f7d081f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080c4030 0x7f7d081f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49138/0 (socket says 172.21.15.88:49138) 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d0659c640 1 -- 172.21.15.88:0/3807287704 learned_addr learned my addr 172.21.15.88:0/3807287704 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d05d9b640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d080c49f0 0x7f7d081562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.629+0000 7f7d06d9d640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d08150220 0x7f7d081f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0659c640 1 -- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d08150220 msgr2=0x7f7d081f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0659c640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d08150220 0x7f7d081f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0659c640 1 -- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d080c49f0 msgr2=0x7f7d081562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0659c640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d080c49f0 0x7f7d081562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0659c640 1 -- 172.21.15.88:0/3807287704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d081f2750 con 0x7f7d080c4030 2024-09-21T02:07:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0659c640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080c4030 0x7f7d081f2210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8034040 tx=0x7f7cf80329a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8041020 con 0x7f7d080c4030 2024-09-21T02:07:19.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf8004040 con 0x7f7d080c4030 2024-09-21T02:07:19.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d081f4630 con 0x7f7d080c4030 2024-09-21T02:07:19.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8036040 con 0x7f7d080c4030 2024-09-21T02:07:19.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.630+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081f1cf0 con 0x7f7d080c4030 2024-09-21T02:07:19.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.631+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d0815a350 con 0x7f7d080c4030 2024-09-21T02:07:19.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.632+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7cf8044020 con 0x7f7d080c4030 2024-09-21T02:07:19.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.632+0000 7f7cf6ffd640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7ce8078bc0 0x7f7ce807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.632+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7cf80bc050 con 0x7f7d080c4030 2024-09-21T02:07:19.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.632+0000 7f7cf6ffd640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f7ce8080f90 0x7f7ce8083460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.632+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 --> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cf8009820 con 0x7f7ce8080f90 2024-09-21T02:07:19.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.632+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7cf80bc4e0 con 0x7f7d080c4030 2024-09-21T02:07:19.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.632+0000 7f7d05d9b640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7ce8078bc0 0x7f7ce807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.633+0000 7f7d06d9d640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f7ce8080f90 0x7f7ce8083460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.633+0000 7f7d06d9d640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f7ce8080f90 0x7f7ce8083460 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.633+0000 7f7d05d9b640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7ce8078bc0 0x7f7ce807b0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d080c9590 tx=0x7f7cfc001d50 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.634+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== osd.2 v2:172.21.15.88:6808/1554825340 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7cf8009820 con 0x7f7ce8080f90 2024-09-21T02:07:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.646+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 --> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7d080ceb70 con 0x7f7ce8080f90 2024-09-21T02:07:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.646+0000 7f7cf6ffd640 1 -- 172.21.15.88:0/3807287704 <== osd.2 v2:172.21.15.88:6808/1554825340 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7d080ceb70 con 0x7f7ce8080f90 2024-09-21T02:07:19.646 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:19.646 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:19.646 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.646+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f7ce8080f90 msgr2=0x7f7ce8083460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.646+0000 7f7d0cc9b640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:6808/1554825340,v1:172.21.15.88:6809/1554825340] conn(0x7f7ce8080f90 0x7f7ce8083460 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7ce8078bc0 msgr2=0x7f7ce807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7ce8078bc0 0x7f7ce807b0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d080c9590 tx=0x7f7cfc001d50 comp rx=0 tx=0).stop 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080c4030 msgr2=0x7f7d081f2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 --2- 172.21.15.88:0/3807287704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080c4030 0x7f7d081f2210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8034040 tx=0x7f7cf80329a0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d06d9d640 1 -- 172.21.15.88:0/3807287704 reap_dead start 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 shutdown_connections 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 >> 172.21.15.88:0/3807287704 conn(0x7f7d080690f0 msgr2=0x7f7d080c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 shutdown_connections 2024-09-21T02:07:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.647+0000 7f7d0cc9b640 1 -- 172.21.15.88:0/3807287704 wait complete. 2024-09-21T02:07:19.657 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 -- 172.21.15.88:0/2488598909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc0cd140 msgr2=0x7f68cc0caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 --2- 172.21.15.88:0/2488598909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc0cd140 0x7f68cc0caf30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009830 tx=0x7f68c802a510 comp rx=0 tx=0).stop 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 -- 172.21.15.88:0/2488598909 shutdown_connections 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 --2- 172.21.15.88:0/2488598909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68cc0cc050 0x7f68cc0cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 --2- 172.21.15.88:0/2488598909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc0cd140 0x7f68cc0caf30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 -- 172.21.15.88:0/2488598909 >> 172.21.15.88:0/2488598909 conn(0x7f68cc0d16b0 msgr2=0x7f68cc0cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 -- 172.21.15.88:0/2488598909 shutdown_connections 2024-09-21T02:07:19.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.770+0000 7f68d4007640 1 -- 172.21.15.88:0/2488598909 wait complete. 2024-09-21T02:07:19.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.772+0000 7f68d4007640 1 Processor -- start 2024-09-21T02:07:19.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.772+0000 7f68d4007640 1 -- start start 2024-09-21T02:07:19.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.773+0000 7f68d4007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68cc0cc050 0x7f68cc1e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.773+0000 7f68d4007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68cc0cd140 0x7f68cc1e8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.773+0000 7f68d4007640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc15f070 0x7f68cc154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.773+0000 7f68d4007640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0cb470 con 0x7f68cc15f070 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.773+0000 7f68d4007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0cbb30 con 0x7f68cc0cc050 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.773+0000 7f68d4007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc0cc9a0 con 0x7f68cc0cd140 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.773+0000 7f68d25a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc15f070 0x7f68cc154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc15f070 0x7f68cc154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49156/0 (socket says 172.21.15.88:49156) 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 -- 172.21.15.88:0/3015734057 learned_addr learned my addr 172.21.15.88:0/3015734057 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d15a2640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68cc0cd140 0x7f68cc1e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d1da3640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68cc0cc050 0x7f68cc1e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 -- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68cc0cd140 msgr2=0x7f68cc1e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68cc0cd140 0x7f68cc1e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 -- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68cc0cc050 msgr2=0x7f68cc1e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68cc0cc050 0x7f68cc1e6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 -- 172.21.15.88:0/3015734057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc1e6750 con 0x7f68cc15f070 2024-09-21T02:07:19.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d25a4640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc15f070 0x7f68cc154910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f68c400e990 tx=0x7f68c400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68c27fc640 1 -- 172.21.15.88:0/3015734057 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4004d20 con 0x7f68cc15f070 2024-09-21T02:07:19.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68c27fc640 1 -- 172.21.15.88:0/3015734057 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68c400cb00 con 0x7f68cc15f070 2024-09-21T02:07:19.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc0d0960 con 0x7f68cc15f070 2024-09-21T02:07:19.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68c27fc640 1 -- 172.21.15.88:0/3015734057 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4005020 con 0x7f68cc15f070 2024-09-21T02:07:19.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.774+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc0d0c00 con 0x7f68cc15f070 2024-09-21T02:07:19.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.775+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68cc15bbe0 con 0x7f68cc15f070 2024-09-21T02:07:19.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.776+0000 7f68c27fc640 1 -- 172.21.15.88:0/3015734057 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f68c40051c0 con 0x7f68cc15f070 2024-09-21T02:07:19.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.776+0000 7f68c27fc640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68b0078bc0 0x7f68b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:19.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.776+0000 7f68c27fc640 1 -- 172.21.15.88:0/3015734057 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f68c409b610 con 0x7f68cc15f070 2024-09-21T02:07:19.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.777+0000 7f68d1da3640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68b0078bc0 0x7f68b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:19.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.777+0000 7f68d1da3640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68b0078bc0 0x7f68b007b0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68c8034000 tx=0x7f68c8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:19.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.781+0000 7f68c27fc640 1 -- 172.21.15.88:0/3015734057 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68c4016030 con 0x7f68cc15f070 2024-09-21T02:07:19.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.957+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f68cc1f2410 con 0x7f68b0078bc0 2024-09-21T02:07:19.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.958+0000 7f68c27fc640 1 -- 172.21.15.88:0/3015734057 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f68cc1f2410 con 0x7f68b0078bc0 2024-09-21T02:07:19.958 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T02:07:19.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68b0078bc0 msgr2=0x7f68b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68b0078bc0 0x7f68b007b0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68c8034000 tx=0x7f68c8004730 comp rx=0 tx=0).stop 2024-09-21T02:07:19.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc15f070 msgr2=0x7f68cc154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:19.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc15f070 0x7f68cc154910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f68c400e990 tx=0x7f68c400ee50 comp rx=0 tx=0).stop 2024-09-21T02:07:19.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 shutdown_connections 2024-09-21T02:07:19.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f68b0078bc0 0x7f68b007b0d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68cc15f070 0x7f68cc154910 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68cc0cd140 0x7f68cc1e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 --2- 172.21.15.88:0/3015734057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68cc0cc050 0x7f68cc1e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:19.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 >> 172.21.15.88:0/3015734057 conn(0x7f68cc0d16b0 msgr2=0x7f68cc15abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:19.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 shutdown_connections 2024-09-21T02:07:19.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:19.960+0000 7f68d4007640 1 -- 172.21.15.88:0/3015734057 wait complete. 2024-09-21T02:07:19.971 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T02:07:19.972 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 -- 172.21.15.88:0/3042830801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 msgr2=0x7fe270156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 --2- 172.21.15.88:0/3042830801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 0x7fe270156930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe268005fd0 tx=0x7fe26802ab20 comp rx=0 tx=0).stop 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 -- 172.21.15.88:0/3042830801 shutdown_connections 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 --2- 172.21.15.88:0/3042830801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 0x7fe270156930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 --2- 172.21.15.88:0/3042830801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2700c7c20 0x7fe27006eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 -- 172.21.15.88:0/3042830801 >> 172.21.15.88:0/3042830801 conn(0x7fe27006de70 msgr2=0x7fe27006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 -- 172.21.15.88:0/3042830801 shutdown_connections 2024-09-21T02:07:20.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.088+0000 7fe275501640 1 -- 172.21.15.88:0/3042830801 wait complete. 2024-09-21T02:07:20.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.090+0000 7fe275501640 1 Processor -- start 2024-09-21T02:07:20.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.090+0000 7fe275501640 1 -- start start 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.091+0000 7fe275501640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 0x7fe2700ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.091+0000 7fe275501640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2700c7c20 0x7fe2700abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.091+0000 7fe275501640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2700cc440 0x7fe2701507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.091+0000 7fe275501640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2700c8840 con 0x7fe2700707b0 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.091+0000 7fe275501640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2700c92d0 con 0x7fe2700cc440 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.091+0000 7fe275501640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fe270074db0 con 0x7fe2700c7c20 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.091+0000 7fe26effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 0x7fe2700ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe26effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 0x7fe2700ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49178/0 (socket says 172.21.15.88:49178) 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe26effd640 1 -- 172.21.15.88:0/3618275618 learned_addr learned my addr 172.21.15.88:0/3618275618 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe267fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2700c7c20 0x7fe2700abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe26f7fe640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2700cc440 0x7fe2701507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe267fff640 1 -- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2700cc440 msgr2=0x7fe2701507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe267fff640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2700cc440 0x7fe2701507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe267fff640 1 -- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 msgr2=0x7fe2700ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe267fff640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 0x7fe2700ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe267fff640 1 -- 172.21.15.88:0/3618275618 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2700ac440 con 0x7fe2700c7c20 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe26effd640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2700707b0 0x7fe2700ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:07:20.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.092+0000 7fe267fff640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2700c7c20 0x7fe2700abf00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e6a0 tx=0x7fe25800eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.093+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258004220 con 0x7fe2700c7c20 2024-09-21T02:07:20.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.093+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2580043c0 con 0x7fe2700c7c20 2024-09-21T02:07:20.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.093+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2701f00f0 con 0x7fe2700c7c20 2024-09-21T02:07:20.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.093+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25800cb10 con 0x7fe2700c7c20 2024-09-21T02:07:20.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.093+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe270158da0 con 0x7fe2700c7c20 2024-09-21T02:07:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.093+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2700c97f0 con 0x7fe2700c7c20 2024-09-21T02:07:20.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.094+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe25800ccb0 con 0x7fe2700c7c20 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe2677fe640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe250078bc0 0x7fe25007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe25809b9e0 con 0x7fe2700c7c20 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe26effd640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe250078bc0 0x7fe25007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe2677fe640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7fe250080f90 0x7fe250083460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 --> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe25800fe50 con 0x7fe250080f90 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe25809be40 con 0x7fe2700c7c20 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe26f7fe640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7fe250080f90 0x7fe250083460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe26effd640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe250078bc0 0x7fe25007b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe260001bd0 tx=0x7fe260009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.095+0000 7fe26f7fe640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7fe250080f90 0x7fe250083460 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.096+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== osd.3 v2:172.21.15.88:6816/723368200 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe25800fe50 con 0x7fe250080f90 2024-09-21T02:07:20.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.108+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 --> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe2700758d0 con 0x7fe250080f90 2024-09-21T02:07:20.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe2677fe640 1 -- 172.21.15.88:0/3618275618 <== osd.3 v2:172.21.15.88:6816/723368200 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe2700758d0 con 0x7fe250080f90 2024-09-21T02:07:20.109 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:20.109 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:20.109 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7fe250080f90 msgr2=0x7fe250083460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.88:6816/723368200,v1:172.21.15.88:6817/723368200] conn(0x7fe250080f90 0x7fe250083460 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe250078bc0 msgr2=0x7fe25007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fe250078bc0 0x7fe25007b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe260001bd0 tx=0x7fe260009040 comp rx=0 tx=0).stop 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2700c7c20 msgr2=0x7fe2700abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 --2- 172.21.15.88:0/3618275618 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2700c7c20 0x7fe2700abf00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e6a0 tx=0x7fe25800eb60 comp rx=0 tx=0).stop 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe26f7fe640 1 -- 172.21.15.88:0/3618275618 reap_dead start 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 shutdown_connections 2024-09-21T02:07:20.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 >> 172.21.15.88:0/3618275618 conn(0x7fe27006de70 msgr2=0x7fe270152b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 shutdown_connections 2024-09-21T02:07:20.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.109+0000 7fe275501640 1 -- 172.21.15.88:0/3618275618 wait complete. 2024-09-21T02:07:20.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T02:07:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.234+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3934008863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 msgr2=0x7f9e9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.234+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3934008863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9e9400b9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001d90 tx=0x7f9e94005830 comp rx=0 tx=0).stop 2024-09-21T02:07:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.234+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3934008863 shutdown_connections 2024-09-21T02:07:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.234+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3934008863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9e9400b9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.234+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3934008863 >> 172.21.15.88:0/3934008863 conn(0x7f9ea80d16b0 msgr2=0x7f9ea80cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.234+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3934008863 shutdown_connections 2024-09-21T02:07:20.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.234+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3934008863 wait complete. 2024-09-21T02:07:20.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.236+0000 7f9eb01c3640 1 Processor -- start 2024-09-21T02:07:20.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.236+0000 7f9eb01c3640 1 -- start start 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eb01c3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9ea81f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eb01c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea80c4b90 0x7f9ea80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eb01c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea80c7510 0x7f9ea80c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eb01c3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea815bb00 con 0x7f9e94005250 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eb01c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea80cbb30 con 0x7f9ea80c4b90 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eb01c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea815bc80 con 0x7f9ea80c7510 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eadf5f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9ea81f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eadf5f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9ea81f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49190/0 (socket says 172.21.15.88:49190) 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9eadf5f640 1 -- 172.21.15.88:0/3007311954 learned_addr learned my addr 172.21.15.88:0/3007311954 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.237+0000 7f9ead75e640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea80c4b90 0x7f9ea80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eae760640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea80c7510 0x7f9ea80c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eadf5f640 1 -- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea80c7510 msgr2=0x7f9ea80c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eadf5f640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea80c7510 0x7f9ea80c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eadf5f640 1 -- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea80c4b90 msgr2=0x7f9ea80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eadf5f640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea80c4b90 0x7f9ea80cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eadf5f640 1 -- 172.21.15.88:0/3007311954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea8156470 con 0x7f9e94005250 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eadf5f640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9ea81f2440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006680 tx=0x7f9ea40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9e9e7fc640 1 -- 172.21.15.88:0/3007311954 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4015070 con 0x7f9e94005250 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9e9e7fc640 1 -- 172.21.15.88:0/3007311954 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ea4011860 con 0x7f9e94005250 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9e9e7fc640 1 -- 172.21.15.88:0/3007311954 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4011cb0 con 0x7f9e94005250 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81f2980 con 0x7f9e94005250 2024-09-21T02:07:20.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.238+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea80c6740 con 0x7f9e94005250 2024-09-21T02:07:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.239+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea8156640 con 0x7f9e94005250 2024-09-21T02:07:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.240+0000 7f9e9e7fc640 1 -- 172.21.15.88:0/3007311954 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ea4018070 con 0x7f9e94005250 2024-09-21T02:07:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.240+0000 7f9e9e7fc640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.241+0000 7f9e9e7fc640 1 -- 172.21.15.88:0/3007311954 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9ea409ba90 con 0x7f9e94005250 2024-09-21T02:07:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.241+0000 7f9ead75e640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.241+0000 7f9ead75e640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ea815b1b0 tx=0x7f9e98005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.245+0000 7f9e9e7fc640 1 -- 172.21.15.88:0/3007311954 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ea40638a0 con 0x7f9e94005250 2024-09-21T02:07:20.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.425+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f9ea80ce880 con 0x7f9e8c078b90 2024-09-21T02:07:20.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.426+0000 7f9e9e7fc640 1 -- 172.21.15.88:0/3007311954 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9ea80ce880 con 0x7f9e8c078b90 2024-09-21T02:07:20.426 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T02:07:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9e8c078b90 msgr2=0x7f9e8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ea815b1b0 tx=0x7f9e98005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 msgr2=0x7f9ea81f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9ea81f2440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006680 tx=0x7f9ea40039c0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 shutdown_connections 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f9e8c078b90 0x7f9e8c07b0a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea80c7510 0x7f9ea80c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea80c4b90 0x7f9ea80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 --2- 172.21.15.88:0/3007311954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e94005250 0x7f9ea81f2440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 >> 172.21.15.88:0/3007311954 conn(0x7f9ea80d16b0 msgr2=0x7f9ea80cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 shutdown_connections 2024-09-21T02:07:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.428+0000 7f9eb01c3640 1 -- 172.21.15.88:0/3007311954 wait complete. 2024-09-21T02:07:20.440 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T02:07:20.440 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:20.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 -- 172.21.15.88:0/1924697795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0c4130 msgr2=0x7fcd7c153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 --2- 172.21.15.88:0/1924697795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0c4130 0x7fcd7c153720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005fd0 tx=0x7fcd7802a8b0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 -- 172.21.15.88:0/1924697795 shutdown_connections 2024-09-21T02:07:20.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 --2- 172.21.15.88:0/1924697795 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7c0c9b10 0x7fcd7c0c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 --2- 172.21.15.88:0/1924697795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0c4130 0x7fcd7c153720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 -- 172.21.15.88:0/1924697795 >> 172.21.15.88:0/1924697795 conn(0x7fcd7c068d00 msgr2=0x7fcd7c069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 -- 172.21.15.88:0/1924697795 shutdown_connections 2024-09-21T02:07:20.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.557+0000 7fcd84215640 1 -- 172.21.15.88:0/1924697795 wait complete. 2024-09-21T02:07:20.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.559+0000 7fcd84215640 1 Processor -- start 2024-09-21T02:07:20.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.559+0000 7fcd84215640 1 -- start start 2024-09-21T02:07:20.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd84215640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c0c4130 0x7fcd7c1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd84215640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7c0c9b10 0x7fcd7c1df810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd84215640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0cbba0 0x7fcd7c1dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd84215640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c15a190 con 0x7fcd7c0c4130 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd84215640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c0cb620 con 0x7fcd7c0cbba0 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd84215640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7c15a310 con 0x7fcd7c0c9b10 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c0c4130 0x7fcd7c1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c0c4130 0x7fcd7c1509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49194/0 (socket says 172.21.15.88:49194) 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 -- 172.21.15.88:0/2164307827 learned_addr learned my addr 172.21.15.88:0/2164307827 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd827b2640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0cbba0 0x7fcd7c1dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd817b0640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7c0c9b10 0x7fcd7c1df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 -- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7c0c9b10 msgr2=0x7fcd7c1df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7c0c9b10 0x7fcd7c1df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 -- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0cbba0 msgr2=0x7fcd7c1dfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0cbba0 0x7fcd7c1dfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd81fb1640 1 -- 172.21.15.88:0/2164307827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c1ef640 con 0x7fcd7c0c4130 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd827b2640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd7c0cbba0 0x7fcd7c1dfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.560+0000 7fcd817b0640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7c0c9b10 0x7fcd7c1df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.561+0000 7fcd81fb1640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c0c4130 0x7fcd7c1509b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009cd0 tx=0x7fcd780316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.561+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78031990 con 0x7fcd7c0c4130 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.561+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd78031c20 con 0x7fcd7c0c4130 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.561+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c0b1a60 con 0x7fcd7c0c4130 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.561+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd780330d0 con 0x7fcd7c0c4130 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.561+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c1efba0 con 0x7fcd7c0c4130 2024-09-21T02:07:20.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.562+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd7c15a8f0 con 0x7fcd7c0c4130 2024-09-21T02:07:20.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.562+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd78033270 con 0x7fcd7c0c4130 2024-09-21T02:07:20.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.563+0000 7fcd727fc640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcd60078b90 0x7fcd6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.563+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcd780bc610 con 0x7fcd7c0c4130 2024-09-21T02:07:20.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.563+0000 7fcd817b0640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcd60078b90 0x7fcd6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.563+0000 7fcd727fc640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7fcd60080f60 0x7fcd60083430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.563+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 --> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd780317e0 con 0x7fcd60080f60 2024-09-21T02:07:20.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.563+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcd780bcaa0 con 0x7fcd7c0c4130 2024-09-21T02:07:20.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.564+0000 7fcd827b2640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7fcd60080f60 0x7fcd60083430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.564+0000 7fcd817b0640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcd60078b90 0x7fcd6007b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0d0120 tx=0x7fcd6c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.564+0000 7fcd827b2640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7fcd60080f60 0x7fcd60083430 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.565+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== osd.4 v2:172.21.15.195:6818/2209751664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd780317e0 con 0x7fcd60080f60 2024-09-21T02:07:20.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.577+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 --> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcd7c153720 con 0x7fcd60080f60 2024-09-21T02:07:20.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.577+0000 7fcd727fc640 1 -- 172.21.15.88:0/2164307827 <== osd.4 v2:172.21.15.195:6818/2209751664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcd7c153720 con 0x7fcd60080f60 2024-09-21T02:07:20.577 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:20.578 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:20.578 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7fcd60080f60 msgr2=0x7fcd60083430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6818/2209751664,v1:172.21.15.195:6819/2209751664] conn(0x7fcd60080f60 0x7fcd60083430 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcd60078b90 msgr2=0x7fcd6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fcd60078b90 0x7fcd6007b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0d0120 tx=0x7fcd6c005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c0c4130 msgr2=0x7fcd7c1509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 --2- 172.21.15.88:0/2164307827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7c0c4130 0x7fcd7c1509b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009cd0 tx=0x7fcd780316f0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd827b2640 1 -- 172.21.15.88:0/2164307827 reap_dead start 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 shutdown_connections 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 >> 172.21.15.88:0/2164307827 conn(0x7fcd7c068d00 msgr2=0x7fcd7c069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 shutdown_connections 2024-09-21T02:07:20.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.578+0000 7fcd84215640 1 -- 172.21.15.88:0/2164307827 wait complete. 2024-09-21T02:07:20.588 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T02:07:20.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 -- 172.21.15.88:0/2706986211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb780693a0 msgr2=0x7ffb78164180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 --2- 172.21.15.88:0/2706986211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb780693a0 0x7ffb78164180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009ca0 tx=0x7ffb7002a9a0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 -- 172.21.15.88:0/2706986211 shutdown_connections 2024-09-21T02:07:20.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 --2- 172.21.15.88:0/2706986211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb780693a0 0x7ffb78164180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 --2- 172.21.15.88:0/2706986211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb78068d00 0x7ffb7815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 -- 172.21.15.88:0/2706986211 >> 172.21.15.88:0/2706986211 conn(0x7ffb78070960 msgr2=0x7ffb7806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 -- 172.21.15.88:0/2706986211 shutdown_connections 2024-09-21T02:07:20.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.702+0000 7ffb7f625640 1 -- 172.21.15.88:0/2706986211 wait complete. 2024-09-21T02:07:20.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.704+0000 7ffb7f625640 1 Processor -- start 2024-09-21T02:07:20.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.704+0000 7ffb7f625640 1 -- start start 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7f625640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb78068d00 0x7ffb781f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7f625640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb780693a0 0x7ffb780c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7f625640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb78153a70 0x7ffb781569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7f625640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78159ed0 con 0x7ffb78153a70 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7f625640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78159990 con 0x7ffb78068d00 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7f625640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7815a6d0 con 0x7ffb780693a0 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb78153a70 0x7ffb781569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb78153a70 0x7ffb781569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49206/0 (socket says 172.21.15.88:49206) 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 -- 172.21.15.88:0/1779208794 learned_addr learned my addr 172.21.15.88:0/1779208794 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7d3c1640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb78068d00 0x7ffb781f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7cbc0640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb780693a0 0x7ffb780c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 -- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb780693a0 msgr2=0x7ffb780c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb780693a0 0x7ffb780c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 -- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb78068d00 msgr2=0x7ffb781f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb78068d00 0x7ffb781f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.705+0000 7ffb7dbc2640 1 -- 172.21.15.88:0/1779208794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb781f4300 con 0x7ffb78153a70 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.706+0000 7ffb7dbc2640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb78153a70 0x7ffb781569d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffb70006010 tx=0x7ffb70031f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.706+0000 7ffb6dffb640 1 -- 172.21.15.88:0/1779208794 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70034040 con 0x7ffb78153a70 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.706+0000 7ffb6dffb640 1 -- 172.21.15.88:0/1779208794 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb70004250 con 0x7ffb78153a70 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.706+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7814b8f0 con 0x7ffb78153a70 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.706+0000 7ffb6dffb640 1 -- 172.21.15.88:0/1779208794 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70004600 con 0x7ffb78153a70 2024-09-21T02:07:20.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.706+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb781e4610 con 0x7ffb78153a70 2024-09-21T02:07:20.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.707+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb781e4850 con 0x7ffb78153a70 2024-09-21T02:07:20.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.709+0000 7ffb6dffb640 1 -- 172.21.15.88:0/1779208794 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffb70037020 con 0x7ffb78153a70 2024-09-21T02:07:20.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.709+0000 7ffb6dffb640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffb5c078b90 0x7ffb5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:20.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.709+0000 7ffb6dffb640 1 -- 172.21.15.88:0/1779208794 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ffb700bc590 con 0x7ffb78153a70 2024-09-21T02:07:20.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.709+0000 7ffb7d3c1640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffb5c078b90 0x7ffb5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:20.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.710+0000 7ffb7d3c1640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffb5c078b90 0x7ffb5c07b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009b30 tx=0x7ffb74005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:20.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.713+0000 7ffb6dffb640 1 -- 172.21.15.88:0/1779208794 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffb781e4850 con 0x7ffb78153a70 2024-09-21T02:07:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.889+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7ffb780c5f90 con 0x7ffb5c078b90 2024-09-21T02:07:20.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.890+0000 7ffb6dffb640 1 -- 172.21.15.88:0/1779208794 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffb780c5f90 con 0x7ffb5c078b90 2024-09-21T02:07:20.890 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T02:07:20.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffb5c078b90 msgr2=0x7ffb5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffb5c078b90 0x7ffb5c07b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009b30 tx=0x7ffb74005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:20.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb78153a70 msgr2=0x7ffb781569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:20.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb78153a70 0x7ffb781569d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffb70006010 tx=0x7ffb70031f00 comp rx=0 tx=0).stop 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 shutdown_connections 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7ffb5c078b90 0x7ffb5c07b0a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb78153a70 0x7ffb781569d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb780693a0 0x7ffb780c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 --2- 172.21.15.88:0/1779208794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb78068d00 0x7ffb781f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 >> 172.21.15.88:0/1779208794 conn(0x7ffb78070960 msgr2=0x7ffb781510e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 shutdown_connections 2024-09-21T02:07:20.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:20.892+0000 7ffb7f625640 1 -- 172.21.15.88:0/1779208794 wait complete. 2024-09-21T02:07:20.903 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T02:07:20.903 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.020+0000 7f5bac694640 1 -- 172.21.15.88:0/3475223219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 msgr2=0x7f5ba40cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.020+0000 7f5bac694640 1 --2- 172.21.15.88:0/3475223219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 0x7f5ba40cbba0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5b98005fd0 tx=0x7f5b9802a8b0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.020+0000 7f5bac694640 1 -- 172.21.15.88:0/3475223219 shutdown_connections 2024-09-21T02:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.020+0000 7f5bac694640 1 --2- 172.21.15.88:0/3475223219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 0x7f5ba40cbba0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.020+0000 7f5bac694640 1 -- 172.21.15.88:0/3475223219 >> 172.21.15.88:0/3475223219 conn(0x7f5ba4068d00 msgr2=0x7f5ba4069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.020+0000 7f5bac694640 1 -- 172.21.15.88:0/3475223219 shutdown_connections 2024-09-21T02:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.020+0000 7f5bac694640 1 -- 172.21.15.88:0/3475223219 wait complete. 2024-09-21T02:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.022+0000 7f5bac694640 1 Processor -- start 2024-09-21T02:07:21.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.022+0000 7f5bac694640 1 -- start start 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5bac694640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40c4130 0x7f5ba41f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5bac694640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40cc0e0 0x7f5ba41f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5bac694640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 0x7f5ba41e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5bac694640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40ca220 con 0x7f5ba4153720 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5bac694640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4153b30 con 0x7f5ba40c4130 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5bac694640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba415a190 con 0x7f5ba40cc0e0 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 0x7f5ba41e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 0x7f5ba41e4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49214/0 (socket says 172.21.15.88:49214) 2024-09-21T02:07:21.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 -- 172.21.15.88:0/5305072 learned_addr learned my addr 172.21.15.88:0/5305072 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5ba9c2f640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40cc0e0 0x7f5ba41f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baa430640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40c4130 0x7f5ba41f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 -- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40cc0e0 msgr2=0x7f5ba41f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40cc0e0 0x7f5ba41f04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 -- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40c4130 msgr2=0x7f5ba41f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ba40c4130 0x7f5ba41f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 -- 172.21.15.88:0/5305072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba41f4000 con 0x7f5ba4153720 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5ba9c2f640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ba40cc0e0 0x7f5ba41f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.023+0000 7f5baac31640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 0x7f5ba41e4c70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c3f0 tx=0x7f5b9c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.024+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c017070 con 0x7f5ba4153720 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.024+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b9c0035f0 con 0x7f5ba4153720 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.024+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba41f0a10 con 0x7f5ba4153720 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.024+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c003a40 con 0x7f5ba4153720 2024-09-21T02:07:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.024+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba41e3220 con 0x7f5ba4153720 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.024+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ba41e3460 con 0x7f5ba4153720 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.025+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5b9c01a070 con 0x7f5ba4153720 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.026+0000 7f5ba2ffd640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5b90078b90 0x7f5b9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.026+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5b9c099fd0 con 0x7f5ba4153720 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.026+0000 7f5baa430640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5b90078b90 0x7f5b9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.026+0000 7f5ba2ffd640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f5b90080f60 0x7f5b90083430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.026+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 --> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b9c016ec0 con 0x7f5b90080f60 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.026+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5b9c0a1050 con 0x7f5ba4153720 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.026+0000 7f5ba9c2f640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f5b90080f60 0x7f5b90083430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.027+0000 7f5baa430640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5b90078b90 0x7f5b9007b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b94006fd0 tx=0x7f5b94008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.027+0000 7f5ba9c2f640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f5b90080f60 0x7f5b90083430 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.028+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== osd.5 v2:172.21.15.195:6826/3321376084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b9c016ec0 con 0x7f5b90080f60 2024-09-21T02:07:21.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.040+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 --> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5ba40cc4f0 con 0x7f5b90080f60 2024-09-21T02:07:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5ba2ffd640 1 -- 172.21.15.88:0/5305072 <== osd.5 v2:172.21.15.195:6826/3321376084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5ba40cc4f0 con 0x7f5b90080f60 2024-09-21T02:07:21.041 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:21.041 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:21.041 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f5b90080f60 msgr2=0x7f5b90083430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6826/3321376084,v1:172.21.15.195:6827/3321376084] conn(0x7f5b90080f60 0x7f5b90083430 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5b90078b90 msgr2=0x7f5b9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5b90078b90 0x7f5b9007b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b94006fd0 tx=0x7f5b94008040 comp rx=0 tx=0).stop 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 msgr2=0x7f5ba41e4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 --2- 172.21.15.88:0/5305072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ba4153720 0x7f5ba41e4c70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c3f0 tx=0x7f5b9c00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5baac31640 1 -- 172.21.15.88:0/5305072 reap_dead start 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 shutdown_connections 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 >> 172.21.15.88:0/5305072 conn(0x7f5ba4068d00 msgr2=0x7f5ba4152c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 shutdown_connections 2024-09-21T02:07:21.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.041+0000 7f5bac694640 1 -- 172.21.15.88:0/5305072 wait complete. 2024-09-21T02:07:21.052 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 -- 172.21.15.88:0/1208724077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c5d90 msgr2=0x7f702c0726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 --2- 172.21.15.88:0/1208724077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c5d90 0x7f702c0726d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7014009830 tx=0x7f701402a5a0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 -- 172.21.15.88:0/1208724077 shutdown_connections 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 --2- 172.21.15.88:0/1208724077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c5d90 0x7f702c0726d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 --2- 172.21.15.88:0/1208724077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f702c071f60 0x7f702c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 -- 172.21.15.88:0/1208724077 >> 172.21.15.88:0/1208724077 conn(0x7f702c068d00 msgr2=0x7f702c069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 -- 172.21.15.88:0/1208724077 shutdown_connections 2024-09-21T02:07:21.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.166+0000 7f703164a640 1 -- 172.21.15.88:0/1208724077 wait complete. 2024-09-21T02:07:21.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.168+0000 7f703164a640 1 Processor -- start 2024-09-21T02:07:21.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.168+0000 7f703164a640 1 -- start start 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.169+0000 7f703164a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f702c0c5d90 0x7f702c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f703164a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c8e60 0x7f702c149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f703164a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f702c149f20 0x7f702c1ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f703164a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f702c153cb0 con 0x7f702c0c8e60 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f703164a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f702c159dc0 con 0x7f702c0c5d90 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f703164a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f702c153e30 con 0x7f702c149f20 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c8e60 0x7f702c149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c8e60 0x7f702c149670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49236/0 (socket says 172.21.15.88:49236) 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 -- 172.21.15.88:0/2437970562 learned_addr learned my addr 172.21.15.88:0/2437970562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f702affd640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f702c0c5d90 0x7f702c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f702b7fe640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f702c149f20 0x7f702c1ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 -- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f702c149f20 msgr2=0x7f702c1ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f702c149f20 0x7f702c1ef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 -- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f702c0c5d90 msgr2=0x7f702c155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f702c0c5d90 0x7f702c155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 -- 172.21.15.88:0/2437970562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c155e90 con 0x7f702c0c8e60 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f702b7fe640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f702c149f20 0x7f702c1ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f70227fc640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c8e60 0x7f702c149670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7014009950 tx=0x7f7014033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f7023fff640 1 -- 172.21.15.88:0/2437970562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014031d60 con 0x7f702c0c8e60 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f7023fff640 1 -- 172.21.15.88:0/2437970562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7014033e10 con 0x7f702c0c8e60 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c0cc480 con 0x7f702c0c8e60 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f7023fff640 1 -- 172.21.15.88:0/2437970562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014033310 con 0x7f702c0c8e60 2024-09-21T02:07:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.170+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c14e6f0 con 0x7f702c0c8e60 2024-09-21T02:07:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.171+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702c1f0220 con 0x7f702c0c8e60 2024-09-21T02:07:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.172+0000 7f7023fff640 1 -- 172.21.15.88:0/2437970562 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70140334b0 con 0x7f702c0c8e60 2024-09-21T02:07:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.173+0000 7f7023fff640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f700c078b90 0x7f700c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.173+0000 7f7023fff640 1 -- 172.21.15.88:0/2437970562 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f70140bc0f0 con 0x7f702c0c8e60 2024-09-21T02:07:21.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.173+0000 7f702affd640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f700c078b90 0x7f700c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.173+0000 7f702affd640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f700c078b90 0x7f700c07b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f701c00bb50 tx=0x7f701c00b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.177+0000 7f7023fff640 1 -- 172.21.15.88:0/2437970562 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f702c1f0220 con 0x7f702c0c8e60 2024-09-21T02:07:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.356+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f702c0cc0e0 con 0x7f700c078b90 2024-09-21T02:07:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.357+0000 7f7023fff640 1 -- 172.21.15.88:0/2437970562 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f701c010bd0 con 0x7f700c078b90 2024-09-21T02:07:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T02:07:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f700c078b90 msgr2=0x7f700c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f700c078b90 0x7f700c07b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f701c00bb50 tx=0x7f701c00b2e0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c8e60 msgr2=0x7f702c149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c8e60 0x7f702c149670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7014009950 tx=0x7f7014033c20 comp rx=0 tx=0).stop 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 shutdown_connections 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f700c078b90 0x7f700c07b0a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f702c149f20 0x7f702c1ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c0c8e60 0x7f702c149670 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 --2- 172.21.15.88:0/2437970562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f702c0c5d90 0x7f702c155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 >> 172.21.15.88:0/2437970562 conn(0x7f702c068d00 msgr2=0x7f702c0ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 shutdown_connections 2024-09-21T02:07:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.359+0000 7f703164a640 1 -- 172.21.15.88:0/2437970562 wait complete. 2024-09-21T02:07:21.371 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T02:07:21.371 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1551369941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380705d0 msgr2=0x7f2d380709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 --2- 172.21.15.88:0/1551369941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380705d0 0x7f2d380709e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2d380cbcf0 tx=0x7f2d2802a5f0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1551369941 shutdown_connections 2024-09-21T02:07:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 --2- 172.21.15.88:0/1551369941 >> v2:172.21.15.195:3300/0 conn(0x7f2d380c9380 0x7f2d380c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 --2- 172.21.15.88:0/1551369941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380705d0 0x7f2d380709e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 --2- 172.21.15.88:0/1551369941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d380c2e60 0x7f2d381525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1551369941 >> 172.21.15.88:0/1551369941 conn(0x7f2d3806de70 msgr2=0x7f2d3806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1551369941 shutdown_connections 2024-09-21T02:07:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.489+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1551369941 wait complete. 2024-09-21T02:07:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.490+0000 7f2d3f3c3640 1 Processor -- start 2024-09-21T02:07:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.490+0000 7f2d3f3c3640 1 -- start start 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3f3c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d380705d0 0x7f2d381f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3f3c3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380c2e60 0x7f2d380cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3f3c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380c9380 0x7f2d38153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3f3c3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d380ca150 con 0x7f2d380c2e60 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3f3c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d380c8bc0 con 0x7f2d380c9380 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3f3c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380ca430 con 0x7f2d380705d0 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380c2e60 0x7f2d380cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380c2e60 0x7f2d380cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49240/0 (socket says 172.21.15.88:49240) 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 -- 172.21.15.88:0/1769005622 learned_addr learned my addr 172.21.15.88:0/1769005622 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3d15f640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d380705d0 0x7f2d381f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3d960640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380c9380 0x7f2d38153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 -- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d380705d0 msgr2=0x7f2d381f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d380705d0 0x7f2d381f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 -- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380c9380 msgr2=0x7f2d38153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380c9380 0x7f2d38153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 -- 172.21.15.88:0/1769005622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d381f3ea0 con 0x7f2d380c2e60 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d3c95e640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380c2e60 0x7f2d380cce70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2d28000c00 tx=0x7f2d280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.492+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2803e030 con 0x7f2d380c2e60 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.493+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d28003ca0 con 0x7f2d380c2e60 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.493+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28035040 con 0x7f2d380c2e60 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.493+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d381519c0 con 0x7f2d380c2e60 2024-09-21T02:07:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.493+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d381f3440 con 0x7f2d380c2e60 2024-09-21T02:07:21.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.494+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d381482d0 con 0x7f2d380c2e60 2024-09-21T02:07:21.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.495+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d28041030 con 0x7f2d380c2e60 2024-09-21T02:07:21.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.495+0000 7f2d2dffb640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d1c078b90 0x7f2d1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.495+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2d280bb370 con 0x7f2d380c2e60 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.495+0000 7f2d2dffb640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f2d1c080f00 0x7f2d1c0833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.495+0000 7f2d3d15f640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d1c078b90 0x7f2d1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.495+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 --> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d28045e50 con 0x7f2d1c080f00 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.495+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2d280c2050 con 0x7f2d380c2e60 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.496+0000 7f2d3d960640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f2d1c080f00 0x7f2d1c0833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.496+0000 7f2d3d15f640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d1c078b90 0x7f2d1c07b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009b30 tx=0x7f2d34005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.496+0000 7f2d3d960640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f2d1c080f00 0x7f2d1c0833d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.497+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== osd.6 v2:172.21.15.195:6802/1905698167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d28045e50 con 0x7f2d1c080f00 2024-09-21T02:07:21.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.509+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 --> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2d38151ee0 con 0x7f2d1c080f00 2024-09-21T02:07:21.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d2dffb640 1 -- 172.21.15.88:0/1769005622 <== osd.6 v2:172.21.15.195:6802/1905698167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2d38151ee0 con 0x7f2d1c080f00 2024-09-21T02:07:21.510 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:21.510 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:21.510 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f2d1c080f00 msgr2=0x7f2d1c0833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6802/1905698167,v1:172.21.15.195:6803/1905698167] conn(0x7f2d1c080f00 0x7f2d1c0833d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d1c078b90 msgr2=0x7f2d1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d1c078b90 0x7f2d1c07b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009b30 tx=0x7f2d34005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380c2e60 msgr2=0x7f2d380cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 --2- 172.21.15.88:0/1769005622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d380c2e60 0x7f2d380cce70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2d28000c00 tx=0x7f2d280038a0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3d960640 1 -- 172.21.15.88:0/1769005622 reap_dead start 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 shutdown_connections 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 >> 172.21.15.88:0/1769005622 conn(0x7f2d3806de70 msgr2=0x7f2d38073fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 shutdown_connections 2024-09-21T02:07:21.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.510+0000 7f2d3f3c3640 1 -- 172.21.15.88:0/1769005622 wait complete. 2024-09-21T02:07:21.521 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.633+0000 7f4981da1640 1 -- 172.21.15.88:0/4227040227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c072e40 msgr2=0x7f497c159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.633+0000 7f4981da1640 1 --2- 172.21.15.88:0/4227040227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c072e40 0x7f497c159f80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f496c005fd0 tx=0x7f496c02a620 comp rx=0 tx=0).stop 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.633+0000 7f4981da1640 1 -- 172.21.15.88:0/4227040227 shutdown_connections 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.633+0000 7f4981da1640 1 --2- 172.21.15.88:0/4227040227 >> v2:172.21.15.195:3300/0 conn(0x7f497c075770 0x7f497c1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.633+0000 7f4981da1640 1 --2- 172.21.15.88:0/4227040227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c072e40 0x7f497c159f80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.634+0000 7f4981da1640 1 -- 172.21.15.88:0/4227040227 >> 172.21.15.88:0/4227040227 conn(0x7f497c06d8f0 msgr2=0x7f497c0ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.634+0000 7f4981da1640 1 -- 172.21.15.88:0/4227040227 shutdown_connections 2024-09-21T02:07:21.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.634+0000 7f4981da1640 1 -- 172.21.15.88:0/4227040227 wait complete. 2024-09-21T02:07:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.635+0000 7f4981da1640 1 Processor -- start 2024-09-21T02:07:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.635+0000 7f4981da1640 1 -- start start 2024-09-21T02:07:21.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f4981da1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f497c075770 0x7f497c1f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f4981da1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c153a10 0x7f497c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f4981da1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c0c3260 0x7f497c0c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f4981da1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f497c159780 con 0x7f497c153a10 2024-09-21T02:07:21.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f4981da1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0c9610 con 0x7f497c0c3260 2024-09-21T02:07:21.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f4981da1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0c7110 con 0x7f497c075770 2024-09-21T02:07:21.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c153a10 0x7f497c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c153a10 0x7f497c1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49250/0 (socket says 172.21.15.88:49250) 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 -- 172.21.15.88:0/2978313772 learned_addr learned my addr 172.21.15.88:0/2978313772 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497bfff640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c0c3260 0x7f497c0c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497b7fe640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f497c075770 0x7f497c1f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 -- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f497c075770 msgr2=0x7f497c1f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f497c075770 0x7f497c1f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 -- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c0c3260 msgr2=0x7f497c0c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c0c3260 0x7f497c0c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 -- 172.21.15.88:0/2978313772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c1f5ce0 con 0x7f497c153a10 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.637+0000 7f497affd640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c153a10 0x7f497c1574e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f497000d6a0 tx=0x7f497000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.638+0000 7f4963fff640 1 -- 172.21.15.88:0/2978313772 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970017070 con 0x7f497c153a10 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.638+0000 7f4963fff640 1 -- 172.21.15.88:0/2978313772 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4970009ab0 con 0x7f497c153a10 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.638+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c0c3730 con 0x7f497c153a10 2024-09-21T02:07:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.638+0000 7f4963fff640 1 -- 172.21.15.88:0/2978313772 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970010460 con 0x7f497c153a10 2024-09-21T02:07:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.638+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c0c3cb0 con 0x7f497c153a10 2024-09-21T02:07:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.639+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f497c06e4b0 con 0x7f497c153a10 2024-09-21T02:07:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.639+0000 7f4963fff640 1 -- 172.21.15.88:0/2978313772 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f497001a070 con 0x7f497c153a10 2024-09-21T02:07:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.640+0000 7f4963fff640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4968078bc0 0x7f496807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.640+0000 7f4963fff640 1 -- 172.21.15.88:0/2978313772 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f497009ae70 con 0x7f497c153a10 2024-09-21T02:07:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.640+0000 7f497b7fe640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4968078bc0 0x7f496807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.641+0000 7f497b7fe640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4968078bc0 0x7f496807b0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f496c033000 tx=0x7f496c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.644+0000 7f4963fff640 1 -- 172.21.15.88:0/2978313772 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4970062be0 con 0x7f497c153a10 2024-09-21T02:07:21.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.822+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f497c0cbde0 con 0x7f4968078bc0 2024-09-21T02:07:21.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.823+0000 7f4963fff640 1 -- 172.21.15.88:0/2978313772 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f497c0cbde0 con 0x7f4968078bc0 2024-09-21T02:07:21.823 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T02:07:21.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4968078bc0 msgr2=0x7f496807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4968078bc0 0x7f496807b0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f496c033000 tx=0x7f496c004730 comp rx=0 tx=0).stop 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c153a10 msgr2=0x7f497c1574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c153a10 0x7f497c1574e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f497000d6a0 tx=0x7f497000db60 comp rx=0 tx=0).stop 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 shutdown_connections 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f4968078bc0 0x7f496807b0d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f497c0c3260 0x7f497c0c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f497c153a10 0x7f497c1574e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 --2- 172.21.15.88:0/2978313772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f497c075770 0x7f497c1f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 >> 172.21.15.88:0/2978313772 conn(0x7f497c06d8f0 msgr2=0x7f497c0c8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 shutdown_connections 2024-09-21T02:07:21.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.825+0000 7f4981da1640 1 -- 172.21.15.88:0/2978313772 wait complete. 2024-09-21T02:07:21.836 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T02:07:21.837 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:07:21.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.953+0000 7fad6f182640 1 -- 172.21.15.88:0/2547129838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680c93b0 msgr2=0x7fad68074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.953+0000 7fad6f182640 1 --2- 172.21.15.88:0/2547129838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680c93b0 0x7fad68074490 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fad64009820 tx=0x7fad6402a5a0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.953+0000 7fad6f182640 1 -- 172.21.15.88:0/2547129838 shutdown_connections 2024-09-21T02:07:21.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.953+0000 7fad6f182640 1 --2- 172.21.15.88:0/2547129838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680c93b0 0x7fad68074490 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.953+0000 7fad6f182640 1 -- 172.21.15.88:0/2547129838 >> 172.21.15.88:0/2547129838 conn(0x7fad6806de70 msgr2=0x7fad6806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.953+0000 7fad6f182640 1 -- 172.21.15.88:0/2547129838 shutdown_connections 2024-09-21T02:07:21.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.953+0000 7fad6f182640 1 -- 172.21.15.88:0/2547129838 wait complete. 2024-09-21T02:07:21.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.955+0000 7fad6f182640 1 Processor -- start 2024-09-21T02:07:21.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.955+0000 7fad6f182640 1 -- start start 2024-09-21T02:07:21.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6f182640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad68072220 0x7fad6815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6f182640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad680c8e60 0x7fad68159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6f182640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad680c93b0 0x7fad68144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6f182640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad680cb420 con 0x7fad68072220 2024-09-21T02:07:21.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6f182640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fad680cb2a0 con 0x7fad680c93b0 2024-09-21T02:07:21.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6f182640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fad680c88b0 con 0x7fad680c8e60 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6cf1e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad68072220 0x7fad6815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6cf1e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad68072220 0x7fad6815a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49268/0 (socket says 172.21.15.88:49268) 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6cf1e640 1 -- 172.21.15.88:0/1391047185 learned_addr learned my addr 172.21.15.88:0/1391047185 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad6d71f640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad680c93b0 0x7fad68144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.956+0000 7fad5ffff640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad680c8e60 0x7fad68159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6cf1e640 1 -- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad680c8e60 msgr2=0x7fad68159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6cf1e640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad680c8e60 0x7fad68159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6cf1e640 1 -- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad680c93b0 msgr2=0x7fad68144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6cf1e640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad680c93b0 0x7fad68144730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6cf1e640 1 -- 172.21.15.88:0/1391047185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad68159aa0 con 0x7fad68072220 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6cf1e640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad68072220 0x7fad6815a480 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fad64009940 tx=0x7fad64005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6403f070 con 0x7fad68072220 2024-09-21T02:07:21.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad64004340 con 0x7fad68072220 2024-09-21T02:07:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad681ef580 con 0x7fad68072220 2024-09-21T02:07:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64031bb0 con 0x7fad68072220 2024-09-21T02:07:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.957+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681ef840 con 0x7fad68072220 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.958+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad6815b010 con 0x7fad68072220 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.959+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad64042070 con 0x7fad68072220 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.959+0000 7fad5d7fa640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad54078b90 0x7fad5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.959+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fad640bc3d0 con 0x7fad68072220 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.959+0000 7fad5ffff640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad54078b90 0x7fad5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.959+0000 7fad5d7fa640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7fad54080f60 0x7fad54083430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.959+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 --> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad64046e50 con 0x7fad54080f60 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.959+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fad640bc860 con 0x7fad68072220 2024-09-21T02:07:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.960+0000 7fad6d71f640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7fad54080f60 0x7fad54083430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.960+0000 7fad5ffff640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad54078b90 0x7fad5407b0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad681e9320 tx=0x7fad58009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.960+0000 7fad6d71f640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7fad54080f60 0x7fad54083430 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.961+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== osd.7 v2:172.21.15.195:6810/985762803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad64046e50 con 0x7fad54080f60 2024-09-21T02:07:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.973+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 --> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fad680ca5a0 con 0x7fad54080f60 2024-09-21T02:07:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad5d7fa640 1 -- 172.21.15.88:0/1391047185 <== osd.7 v2:172.21.15.195:6810/985762803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fad680ca5a0 con 0x7fad54080f60 2024-09-21T02:07:21.974 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T02:07:21.974 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:07:21.974 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7fad54080f60 msgr2=0x7fad54083430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6810/985762803,v1:172.21.15.195:6811/985762803] conn(0x7fad54080f60 0x7fad54083430 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad54078b90 msgr2=0x7fad5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fad54078b90 0x7fad5407b0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad681e9320 tx=0x7fad58009040 comp rx=0 tx=0).stop 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad68072220 msgr2=0x7fad6815a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 --2- 172.21.15.88:0/1391047185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad68072220 0x7fad6815a480 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fad64009940 tx=0x7fad64005360 comp rx=0 tx=0).stop 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6d71f640 1 -- 172.21.15.88:0/1391047185 reap_dead start 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 shutdown_connections 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 >> 172.21.15.88:0/1391047185 conn(0x7fad6806de70 msgr2=0x7fad68145a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 shutdown_connections 2024-09-21T02:07:21.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:21.974+0000 7fad6f182640 1 -- 172.21.15.88:0/1391047185 wait complete. 2024-09-21T02:07:21.985 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T02:07:22.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 -- 172.21.15.88:0/4265081182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 msgr2=0x7fa3c01587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/4265081182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 0x7fa3c01587f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00a940 tx=0x7fa3bc02b8f0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 -- 172.21.15.88:0/4265081182 shutdown_connections 2024-09-21T02:07:22.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/4265081182 >> v2:172.21.15.195:3300/0 conn(0x7fa3c0151bc0 0x7fa3c00c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/4265081182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 0x7fa3c01587f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 -- 172.21.15.88:0/4265081182 >> 172.21.15.88:0/4265081182 conn(0x7fa3c006d8f0 msgr2=0x7fa3c00ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:22.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 -- 172.21.15.88:0/4265081182 shutdown_connections 2024-09-21T02:07:22.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.104+0000 7fa3c72bd640 1 -- 172.21.15.88:0/4265081182 wait complete. 2024-09-21T02:07:22.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.105+0000 7fa3c72bd640 1 Processor -- start 2024-09-21T02:07:22.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.106+0000 7fa3c72bd640 1 -- start start 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c72bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 0x7fa3c01e2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c72bd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0151bc0 0x7fa3c00b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c72bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c00cc640 0x7fa3c0156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c72bd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0152460 con 0x7fa3c0151bc0 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c72bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c00c7b80 con 0x7fa3c00cc640 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c72bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c00c85e0 con 0x7fa3c006f2b0 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0151bc0 0x7fa3c00b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0151bc0 0x7fa3c00b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49280/0 (socket says 172.21.15.88:49280) 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 -- 172.21.15.88:0/958351494 learned_addr learned my addr 172.21.15.88:0/958351494 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c585a640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c00cc640 0x7fa3c0156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c5059640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 0x7fa3c01e2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 -- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 msgr2=0x7fa3c01e2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 0x7fa3c01e2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 -- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c00cc640 msgr2=0x7fa3c0156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c00cc640 0x7fa3c0156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 -- 172.21.15.88:0/958351494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c00b0760 con 0x7fa3c0151bc0 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c585a640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c00cc640 0x7fa3c0156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.107+0000 7fa3c4858640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0151bc0 0x7fa3c00b0220 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000ea20 tx=0x7fa3b000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.108+0000 7fa3b5ffb640 1 -- 172.21.15.88:0/958351494 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000cd50 con 0x7fa3c0151bc0 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.108+0000 7fa3b5ffb640 1 -- 172.21.15.88:0/958351494 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3b0004020 con 0x7fa3c0151bc0 2024-09-21T02:07:22.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.108+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c00cb930 con 0x7fa3c0151bc0 2024-09-21T02:07:22.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.108+0000 7fa3b5ffb640 1 -- 172.21.15.88:0/958351494 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0004470 con 0x7fa3c0151bc0 2024-09-21T02:07:22.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.108+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c00cbef0 con 0x7fa3c0151bc0 2024-09-21T02:07:22.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.109+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0071130 con 0x7fa3c0151bc0 2024-09-21T02:07:22.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.109+0000 7fa3b5ffb640 1 -- 172.21.15.88:0/958351494 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3b00035f0 con 0x7fa3c0151bc0 2024-09-21T02:07:22.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.110+0000 7fa3b5ffb640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3a4078bc0 0x7fa3a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.110+0000 7fa3b5ffb640 1 -- 172.21.15.88:0/958351494 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa3b009b630 con 0x7fa3c0151bc0 2024-09-21T02:07:22.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.110+0000 7fa3c5059640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3a4078bc0 0x7fa3a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.111+0000 7fa3c5059640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3a4078bc0 0x7fa3a407b0d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc036000 tx=0x7fa3bc004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:22.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.115+0000 7fa3b5ffb640 1 -- 172.21.15.88:0/958351494 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa3c0071130 con 0x7fa3c0151bc0 2024-09-21T02:07:22.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.291+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa3c0151fd0 con 0x7fa3a4078bc0 2024-09-21T02:07:22.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.292+0000 7fa3b5ffb640 1 -- 172.21.15.88:0/958351494 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa3c0151fd0 con 0x7fa3a4078bc0 2024-09-21T02:07:22.292 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T02:07:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3a4078bc0 msgr2=0x7fa3a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3a4078bc0 0x7fa3a407b0d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc036000 tx=0x7fa3bc004730 comp rx=0 tx=0).stop 2024-09-21T02:07:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0151bc0 msgr2=0x7fa3c00b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0151bc0 0x7fa3c00b0220 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000ea20 tx=0x7fa3b000eee0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 shutdown_connections 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fa3a4078bc0 0x7fa3a407b0d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3c00cc640 0x7fa3c0156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0151bc0 0x7fa3c00b0220 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 --2- 172.21.15.88:0/958351494 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3c006f2b0 0x7fa3c01e2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 >> 172.21.15.88:0/958351494 conn(0x7fa3c006d8f0 msgr2=0x7fa3c00ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 shutdown_connections 2024-09-21T02:07:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.294+0000 7fa3c72bd640 1 -- 172.21.15.88:0/958351494 wait complete. 2024-09-21T02:07:22.306 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.419+0000 7f65462e5640 1 -- 172.21.15.88:0/812993004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 msgr2=0x7f6528003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.419+0000 7f65462e5640 1 --2- 172.21.15.88:0/812993004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 0x7f6528003ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6530001690 tx=0x7f6528005830 comp rx=0 tx=0).stop 2024-09-21T02:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.419+0000 7f6544882640 1 -- 172.21.15.88:0/812993004 reap_dead start 2024-09-21T02:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.419+0000 7f65462e5640 1 -- 172.21.15.88:0/812993004 shutdown_connections 2024-09-21T02:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.419+0000 7f65462e5640 1 -- 172.21.15.88:0/812993004 >> 172.21.15.88:0/812993004 conn(0x7f65400ca8d0 msgr2=0x7f65400cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.419+0000 7f65462e5640 1 -- 172.21.15.88:0/812993004 shutdown_connections 2024-09-21T02:07:22.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.419+0000 7f65462e5640 1 -- 172.21.15.88:0/812993004 wait complete. 2024-09-21T02:07:22.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.420+0000 7f65462e5640 1 Processor -- start 2024-09-21T02:07:22.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.421+0000 7f65462e5640 1 -- start start 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f65462e5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 0x7f654015abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f65462e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6540159b50 0x7f65401f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f65462e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f654006cf40 0x7f654014d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f65462e5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65400cb770 con 0x7f6528003a30 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f65462e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6540158a40 con 0x7f654006cf40 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f65462e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f65400759e0 con 0x7f6540159b50 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 0x7f654015abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 0x7f654015abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49302/0 (socket says 172.21.15.88:49302) 2024-09-21T02:07:22.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 -- 172.21.15.88:0/1947464344 learned_addr learned my addr 172.21.15.88:0/1947464344 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653f7fe640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6540159b50 0x7f65401f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f6544882640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f654006cf40 0x7f654014d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 -- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6540159b50 msgr2=0x7f65401f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6540159b50 0x7f65401f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 -- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f654006cf40 msgr2=0x7f654014d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f654006cf40 0x7f654014d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 -- 172.21.15.88:0/1947464344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65401f38a0 con 0x7f6528003a30 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f6544882640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f654006cf40 0x7f654014d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653ffff640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 0x7f654015abc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6530001780 tx=0x7f6530006270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.422+0000 7f653cff9640 1 -- 172.21.15.88:0/1947464344 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530041070 con 0x7f6528003a30 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.423+0000 7f653cff9640 1 -- 172.21.15.88:0/1947464344 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65300035f0 con 0x7f6528003a30 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.423+0000 7f653cff9640 1 -- 172.21.15.88:0/1947464344 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300039e0 con 0x7f6528003a30 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.423+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654014e7d0 con 0x7f6528003a30 2024-09-21T02:07:22.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.423+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65401f03c0 con 0x7f6528003a30 2024-09-21T02:07:22.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.424+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65401f3a70 con 0x7f6528003a30 2024-09-21T02:07:22.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.424+0000 7f653cff9640 1 -- 172.21.15.88:0/1947464344 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6530044070 con 0x7f6528003a30 2024-09-21T02:07:22.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.425+0000 7f653cff9640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f652c078bc0 0x7f652c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:22.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.425+0000 7f653cff9640 1 -- 172.21.15.88:0/1947464344 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f65300ba110 con 0x7f6528003a30 2024-09-21T02:07:22.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.425+0000 7f653f7fe640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f652c078bc0 0x7f652c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:22.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.426+0000 7f653f7fe640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f652c078bc0 0x7f652c07b0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f65401ed9c0 tx=0x7f6534005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:22.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.430+0000 7f653cff9640 1 -- 172.21.15.88:0/1947464344 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65300824d0 con 0x7f6528003a30 2024-09-21T02:07:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.607+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6540151ec0 con 0x7f652c078bc0 2024-09-21T02:07:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.612+0000 7f653cff9640 1 -- 172.21.15.88:0/1947464344 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253085 (secure 0 0 0) 0x7f653405ecb0 con 0x7f652c078bc0 2024-09-21T02:07:22.615 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:22.616 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:07:22.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f652c078bc0 msgr2=0x7f652c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f652c078bc0 0x7f652c07b0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f65401ed9c0 tx=0x7f6534005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:22.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 msgr2=0x7f654015abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:22.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 0x7f654015abc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6530001780 tx=0x7f6530006270 comp rx=0 tx=0).stop 2024-09-21T02:07:22.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 shutdown_connections 2024-09-21T02:07:22.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f652c078bc0 0x7f652c07b0d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f654006cf40 0x7f654014d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6540159b50 0x7f65401f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 --2- 172.21.15.88:0/1947464344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6528003a30 0x7f654015abc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:22.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.615+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 >> 172.21.15.88:0/1947464344 conn(0x7f65400ca8d0 msgr2=0x7f6540071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:22.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.616+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 shutdown_connections 2024-09-21T02:07:22.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:22.616+0000 7f65462e5640 1 -- 172.21.15.88:0/1947464344 wait complete. 2024-09-21T02:07:22.627 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-21T02:07:20.875729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8214323,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":13143,"num_write_kb":25178470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15131,"ondisk_log_size":15131,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347948,"kb_used_data":18028,"kb_used_omap":12,"kb_used_meta":329779,"kb_avail":754626772,"statfs":{"total":773094113280,"available":772737814528,"internally_reserved":0,"allocated":18460672,"data_stored":17107893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":337694258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003161"},"pg_stats":[{"pgid":"3.3f","version":"23'167","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.930261+0000","last_change":"2024-09-21T02:03:58.087254+0000","last_active":"2024-09-21T02:07:15.930261+0000","last_peered":"2024-09-21T02:07:15.930261+0000","last_clean":"2024-09-21T02:07:15.930261+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:07:15.930261+0000","last_undegraded":"2024-09-21T02:07:15.930261+0000","last_fullsized":"2024-09-21T02:07:15.930261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.281926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'152","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.894759+0000","last_change":"2024-09-21T02:03:58.082819+0000","last_active":"2024-09-21T02:07:15.894759+0000","last_peered":"2024-09-21T02:07:15.894759+0000","last_clean":"2024-09-21T02:07:15.894759+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:07:15.894759+0000","last_undegraded":"2024-09-21T02:07:15.894759+0000","last_fullsized":"2024-09-21T02:07:15.894759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:57.726151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'178","reported_seq":196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.912443+0000","last_change":"2024-09-21T02:03:58.077130+0000","last_active":"2024-09-21T02:07:15.912443+0000","last_peered":"2024-09-21T02:07:15.912443+0000","last_clean":"2024-09-21T02:07:15.912443+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:07:15.912443+0000","last_undegraded":"2024-09-21T02:07:15.912443+0000","last_fullsized":"2024-09-21T02:07:15.912443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:49.042428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'193","reported_seq":211,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934380+0000","last_change":"2024-09-21T02:03:58.080299+0000","last_active":"2024-09-21T02:07:15.934380+0000","last_peered":"2024-09-21T02:07:15.934380+0000","last_clean":"2024-09-21T02:07:15.934380+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:07:15.934380+0000","last_undegraded":"2024-09-21T02:07:15.934380+0000","last_fullsized":"2024-09-21T02:07:15.934380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:00:56.793403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'174","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.924744+0000","last_change":"2024-09-21T02:03:58.086928+0000","last_active":"2024-09-21T02:07:15.924744+0000","last_peered":"2024-09-21T02:07:15.924744+0000","last_clean":"2024-09-21T02:07:15.924744+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:07:15.924744+0000","last_undegraded":"2024-09-21T02:07:15.924744+0000","last_fullsized":"2024-09-21T02:07:15.924744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:38.734625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.920524+0000","last_change":"2024-09-21T02:03:58.082620+0000","last_active":"2024-09-21T02:07:15.920524+0000","last_peered":"2024-09-21T02:07:15.920524+0000","last_clean":"2024-09-21T02:07:15.920524+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:07:15.920524+0000","last_undegraded":"2024-09-21T02:07:15.920524+0000","last_fullsized":"2024-09-21T02:07:15.920524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:38.049910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'188","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.925429+0000","last_change":"2024-09-21T02:03:58.085974+0000","last_active":"2024-09-21T02:07:15.925429+0000","last_peered":"2024-09-21T02:07:15.925429+0000","last_clean":"2024-09-21T02:07:15.925429+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:07:15.925429+0000","last_undegraded":"2024-09-21T02:07:15.925429+0000","last_fullsized":"2024-09-21T02:07:15.925429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:22.371047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'204","reported_seq":222,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936037+0000","last_change":"2024-09-21T02:03:58.082252+0000","last_active":"2024-09-21T02:07:15.936037+0000","last_peered":"2024-09-21T02:07:15.936037+0000","last_clean":"2024-09-21T02:07:15.936037+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:07:15.936037+0000","last_undegraded":"2024-09-21T02:07:15.936037+0000","last_fullsized":"2024-09-21T02:07:15.936037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.831648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'178","reported_seq":196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936803+0000","last_change":"2024-09-21T02:03:58.087942+0000","last_active":"2024-09-21T02:07:15.936803+0000","last_peered":"2024-09-21T02:07:15.936803+0000","last_clean":"2024-09-21T02:07:15.936803+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:07:15.936803+0000","last_undegraded":"2024-09-21T02:07:15.936803+0000","last_fullsized":"2024-09-21T02:07:15.936803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:42.391039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'190","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934828+0000","last_change":"2024-09-21T02:03:58.081042+0000","last_active":"2024-09-21T02:07:15.934828+0000","last_peered":"2024-09-21T02:07:15.934828+0000","last_clean":"2024-09-21T02:07:15.934828+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:07:15.934828+0000","last_undegraded":"2024-09-21T02:07:15.934828+0000","last_fullsized":"2024-09-21T02:07:15.934828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:11.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'214","reported_seq":232,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934153+0000","last_change":"2024-09-21T02:03:58.081627+0000","last_active":"2024-09-21T02:07:15.934153+0000","last_peered":"2024-09-21T02:07:15.934153+0000","last_clean":"2024-09-21T02:07:15.934153+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:07:15.934153+0000","last_undegraded":"2024-09-21T02:07:15.934153+0000","last_fullsized":"2024-09-21T02:07:15.934153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:18:21.185319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'166","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932783+0000","last_change":"2024-09-21T02:03:58.080525+0000","last_active":"2024-09-21T02:07:15.932783+0000","last_peered":"2024-09-21T02:07:15.932783+0000","last_clean":"2024-09-21T02:07:15.932783+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:07:15.932783+0000","last_undegraded":"2024-09-21T02:07:15.932783+0000","last_fullsized":"2024-09-21T02:07:15.932783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:22.956949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.927286+0000","last_change":"2024-09-21T02:03:58.079212+0000","last_active":"2024-09-21T02:07:15.927286+0000","last_peered":"2024-09-21T02:07:15.927286+0000","last_clean":"2024-09-21T02:07:15.927286+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:07:15.927286+0000","last_undegraded":"2024-09-21T02:07:15.927286+0000","last_fullsized":"2024-09-21T02:07:15.927286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:58:05.639795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'172","reported_seq":190,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.914562+0000","last_change":"2024-09-21T02:03:58.074145+0000","last_active":"2024-09-21T02:07:15.914562+0000","last_peered":"2024-09-21T02:07:15.914562+0000","last_clean":"2024-09-21T02:07:15.914562+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:07:15.914562+0000","last_undegraded":"2024-09-21T02:07:15.914562+0000","last_fullsized":"2024-09-21T02:07:15.914562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:11.463275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.901170+0000","last_change":"2024-09-21T02:03:58.070014+0000","last_active":"2024-09-21T02:07:15.901170+0000","last_peered":"2024-09-21T02:07:15.901170+0000","last_clean":"2024-09-21T02:07:15.901170+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:07:15.901170+0000","last_undegraded":"2024-09-21T02:07:15.901170+0000","last_fullsized":"2024-09-21T02:07:15.901170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:56.693948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.921029+0000","last_change":"2024-09-21T02:03:58.085884+0000","last_active":"2024-09-21T02:07:15.921029+0000","last_peered":"2024-09-21T02:07:15.921029+0000","last_clean":"2024-09-21T02:07:15.921029+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:07:15.921029+0000","last_undegraded":"2024-09-21T02:07:15.921029+0000","last_fullsized":"2024-09-21T02:07:15.921029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'176","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.929050+0000","last_change":"2024-09-21T02:04:45.212987+0000","last_active":"2024-09-21T02:07:15.929050+0000","last_peered":"2024-09-21T02:07:15.929050+0000","last_clean":"2024-09-21T02:07:15.929050+0000","last_became_active":"2024-09-21T02:04:45.212611+0000","last_became_peered":"2024-09-21T02:04:45.212611+0000","last_unstale":"2024-09-21T02:07:15.929050+0000","last_undegraded":"2024-09-21T02:07:15.929050+0000","last_fullsized":"2024-09-21T02:07:15.929050+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:54.964291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'179","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.902990+0000","last_change":"2024-09-21T02:03:58.087109+0000","last_active":"2024-09-21T02:07:15.902990+0000","last_peered":"2024-09-21T02:07:15.902990+0000","last_clean":"2024-09-21T02:07:15.902990+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:07:15.902990+0000","last_undegraded":"2024-09-21T02:07:15.902990+0000","last_fullsized":"2024-09-21T02:07:15.902990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:20.087753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'205","reported_seq":223,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.927156+0000","last_change":"2024-09-21T02:03:58.078712+0000","last_active":"2024-09-21T02:07:15.927156+0000","last_peered":"2024-09-21T02:07:15.927156+0000","last_clean":"2024-09-21T02:07:15.927156+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:07:15.927156+0000","last_undegraded":"2024-09-21T02:07:15.927156+0000","last_fullsized":"2024-09-21T02:07:15.927156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:20.335158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'179","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.933398+0000","last_change":"2024-09-21T02:03:58.088886+0000","last_active":"2024-09-21T02:07:15.933398+0000","last_peered":"2024-09-21T02:07:15.933398+0000","last_clean":"2024-09-21T02:07:15.933398+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:07:15.933398+0000","last_undegraded":"2024-09-21T02:07:15.933398+0000","last_fullsized":"2024-09-21T02:07:15.933398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:01.407361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.928810+0000","last_change":"2024-09-21T02:03:58.082023+0000","last_active":"2024-09-21T02:07:15.928810+0000","last_peered":"2024-09-21T02:07:15.928810+0000","last_clean":"2024-09-21T02:07:15.928810+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:07:15.928810+0000","last_undegraded":"2024-09-21T02:07:15.928810+0000","last_fullsized":"2024-09-21T02:07:15.928810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:12.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'199","reported_seq":211,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.928485+0000","last_change":"2024-09-21T02:04:45.212961+0000","last_active":"2024-09-21T02:07:15.928485+0000","last_peered":"2024-09-21T02:07:15.928485+0000","last_clean":"2024-09-21T02:07:15.928485+0000","last_became_active":"2024-09-21T02:04:45.212587+0000","last_became_peered":"2024-09-21T02:04:45.212587+0000","last_unstale":"2024-09-21T02:07:15.928485+0000","last_undegraded":"2024-09-21T02:07:15.928485+0000","last_fullsized":"2024-09-21T02:07:15.928485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:12.690374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"23'150","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.918536+0000","last_change":"2024-09-21T02:03:58.070285+0000","last_active":"2024-09-21T02:07:15.918536+0000","last_peered":"2024-09-21T02:07:15.918536+0000","last_clean":"2024-09-21T02:07:15.918536+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:07:15.918536+0000","last_undegraded":"2024-09-21T02:07:15.918536+0000","last_fullsized":"2024-09-21T02:07:15.918536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.695003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'207","reported_seq":225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.935036+0000","last_change":"2024-09-21T02:03:58.073388+0000","last_active":"2024-09-21T02:07:15.935036+0000","last_peered":"2024-09-21T02:07:15.935036+0000","last_clean":"2024-09-21T02:07:15.935036+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:07:15.935036+0000","last_undegraded":"2024-09-21T02:07:15.935036+0000","last_fullsized":"2024-09-21T02:07:15.935036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:45.842959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.898591+0000","last_change":"2024-09-21T02:03:58.082630+0000","last_active":"2024-09-21T02:07:15.898591+0000","last_peered":"2024-09-21T02:07:15.898591+0000","last_clean":"2024-09-21T02:07:15.898591+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:07:15.898591+0000","last_undegraded":"2024-09-21T02:07:15.898591+0000","last_fullsized":"2024-09-21T02:07:15.898591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:35.509508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'217","reported_seq":235,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936562+0000","last_change":"2024-09-21T02:03:58.083353+0000","last_active":"2024-09-21T02:07:15.936562+0000","last_peered":"2024-09-21T02:07:15.936562+0000","last_clean":"2024-09-21T02:07:15.936562+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:07:15.936562+0000","last_undegraded":"2024-09-21T02:07:15.936562+0000","last_fullsized":"2024-09-21T02:07:15.936562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:41.304415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'166","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.906274+0000","last_change":"2024-09-21T02:03:58.074499+0000","last_active":"2024-09-21T02:07:15.906274+0000","last_peered":"2024-09-21T02:07:15.906274+0000","last_clean":"2024-09-21T02:07:15.906274+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:07:15.906274+0000","last_undegraded":"2024-09-21T02:07:15.906274+0000","last_fullsized":"2024-09-21T02:07:15.906274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:50.736294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'152","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.920262+0000","last_change":"2024-09-21T02:03:58.081923+0000","last_active":"2024-09-21T02:07:15.920262+0000","last_peered":"2024-09-21T02:07:15.920262+0000","last_clean":"2024-09-21T02:07:15.920262+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:07:15.920262+0000","last_undegraded":"2024-09-21T02:07:15.920262+0000","last_fullsized":"2024-09-21T02:07:15.920262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:26.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'134","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.917423+0000","last_change":"2024-09-21T02:03:58.081260+0000","last_active":"2024-09-21T02:07:15.917423+0000","last_peered":"2024-09-21T02:07:15.917423+0000","last_clean":"2024-09-21T02:07:15.917423+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:07:15.917423+0000","last_undegraded":"2024-09-21T02:07:15.917423+0000","last_fullsized":"2024-09-21T02:07:15.917423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:25.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932066+0000","last_change":"2024-09-21T02:03:58.078629+0000","last_active":"2024-09-21T02:07:15.932066+0000","last_peered":"2024-09-21T02:07:15.932066+0000","last_clean":"2024-09-21T02:07:15.932066+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:07:15.932066+0000","last_undegraded":"2024-09-21T02:07:15.932066+0000","last_fullsized":"2024-09-21T02:07:15.932066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:03.909836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'158","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.904301+0000","last_change":"2024-09-21T02:03:58.077954+0000","last_active":"2024-09-21T02:07:15.904301+0000","last_peered":"2024-09-21T02:07:15.904301+0000","last_clean":"2024-09-21T02:07:15.904301+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:07:15.904301+0000","last_undegraded":"2024-09-21T02:07:15.904301+0000","last_fullsized":"2024-09-21T02:07:15.904301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:31.325451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'180","reported_seq":198,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.923543+0000","last_change":"2024-09-21T02:03:58.085136+0000","last_active":"2024-09-21T02:07:15.923543+0000","last_peered":"2024-09-21T02:07:15.923543+0000","last_clean":"2024-09-21T02:07:15.923543+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:07:15.923543+0000","last_undegraded":"2024-09-21T02:07:15.923543+0000","last_fullsized":"2024-09-21T02:07:15.923543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:51.160972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.910180+0000","last_change":"2024-09-21T02:03:58.067554+0000","last_active":"2024-09-21T02:07:15.910180+0000","last_peered":"2024-09-21T02:07:15.910180+0000","last_clean":"2024-09-21T02:07:15.910180+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:07:15.910180+0000","last_undegraded":"2024-09-21T02:07:15.910180+0000","last_fullsized":"2024-09-21T02:07:15.910180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:33.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1645","reported_seq":1665,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.875706+0000","last_change":"2024-09-21T02:03:57.080834+0000","last_active":"2024-09-21T02:07:15.875706+0000","last_peered":"2024-09-21T02:07:15.875706+0000","last_clean":"2024-09-21T02:07:15.875706+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:07:15.875706+0000","last_undegraded":"2024-09-21T02:07:15.875706+0000","last_fullsized":"2024-09-21T02:07:15.875706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1645,"log_dups_size":0,"ondisk_log_size":1645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:13.513477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3425089,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.923475+0000","last_change":"2024-09-21T02:03:58.080351+0000","last_active":"2024-09-21T02:07:15.923475+0000","last_peered":"2024-09-21T02:07:15.923475+0000","last_clean":"2024-09-21T02:07:15.923475+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:07:15.923475+0000","last_undegraded":"2024-09-21T02:07:15.923475+0000","last_fullsized":"2024-09-21T02:07:15.923475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:55.579365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.279847+0000","last_change":"2024-09-21T02:03:57.078681+0000","last_active":"2024-09-21T02:04:45.279847+0000","last_peered":"2024-09-21T02:04:45.279847+0000","last_clean":"2024-09-21T02:04:45.279847+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:04:45.279847+0000","last_undegraded":"2024-09-21T02:04:45.279847+0000","last_fullsized":"2024-09-21T02:04:45.279847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:02.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280546+0000","last_change":"2024-09-21T02:03:57.083155+0000","last_active":"2024-09-21T02:04:45.280546+0000","last_peered":"2024-09-21T02:04:45.280546+0000","last_clean":"2024-09-21T02:04:45.280546+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:04:45.280546+0000","last_undegraded":"2024-09-21T02:04:45.280546+0000","last_fullsized":"2024-09-21T02:04:45.280546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'190","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.918059+0000","last_change":"2024-09-21T02:03:58.083416+0000","last_active":"2024-09-21T02:07:15.918059+0000","last_peered":"2024-09-21T02:07:15.918059+0000","last_clean":"2024-09-21T02:07:15.918059+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:07:15.918059+0000","last_undegraded":"2024-09-21T02:07:15.918059+0000","last_fullsized":"2024-09-21T02:07:15.918059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:51.287683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205335+0000","last_change":"2024-09-21T02:03:57.072524+0000","last_active":"2024-09-21T02:04:45.205335+0000","last_peered":"2024-09-21T02:04:45.205335+0000","last_clean":"2024-09-21T02:04:45.205335+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:04:45.205335+0000","last_undegraded":"2024-09-21T02:04:45.205335+0000","last_fullsized":"2024-09-21T02:04:45.205335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:03.250376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'187","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.907238+0000","last_change":"2024-09-21T02:03:58.071326+0000","last_active":"2024-09-21T02:07:15.907238+0000","last_peered":"2024-09-21T02:07:15.907238+0000","last_clean":"2024-09-21T02:07:15.907238+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:07:15.907238+0000","last_undegraded":"2024-09-21T02:07:15.907238+0000","last_fullsized":"2024-09-21T02:07:15.907238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:51.048097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.142553+0000","last_change":"2024-09-21T02:03:57.078913+0000","last_active":"2024-09-21T02:05:47.142553+0000","last_peered":"2024-09-21T02:05:47.142553+0000","last_clean":"2024-09-21T02:05:47.142553+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:05:47.142553+0000","last_undegraded":"2024-09-21T02:05:47.142553+0000","last_fullsized":"2024-09-21T02:05:47.142553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:40.284790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'251","reported_seq":263,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.929179+0000","last_change":"2024-09-21T02:04:45.212312+0000","last_active":"2024-09-21T02:07:15.929179+0000","last_peered":"2024-09-21T02:07:15.929179+0000","last_clean":"2024-09-21T02:07:15.929179+0000","last_became_active":"2024-09-21T02:04:45.211981+0000","last_became_peered":"2024-09-21T02:04:45.211981+0000","last_unstale":"2024-09-21T02:07:15.929179+0000","last_undegraded":"2024-09-21T02:07:15.929179+0000","last_fullsized":"2024-09-21T02:07:15.929179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.148945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205736+0000","last_change":"2024-09-21T02:03:57.074881+0000","last_active":"2024-09-21T02:04:45.205736+0000","last_peered":"2024-09-21T02:04:45.205736+0000","last_clean":"2024-09-21T02:04:45.205736+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:04:45.205736+0000","last_undegraded":"2024-09-21T02:04:45.205736+0000","last_fullsized":"2024-09-21T02:04:45.205736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:40.445488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.924039+0000","last_change":"2024-09-21T02:03:58.080875+0000","last_active":"2024-09-21T02:07:15.924039+0000","last_peered":"2024-09-21T02:07:15.924039+0000","last_clean":"2024-09-21T02:07:15.924039+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:07:15.924039+0000","last_undegraded":"2024-09-21T02:07:15.924039+0000","last_fullsized":"2024-09-21T02:07:15.924039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:13.149402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'9","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:45.329388+0000","last_change":"2024-09-21T02:03:57.077836+0000","last_active":"2024-09-21T02:06:45.329388+0000","last_peered":"2024-09-21T02:06:45.329388+0000","last_clean":"2024-09-21T02:06:45.329388+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:06:45.329388+0000","last_undegraded":"2024-09-21T02:06:45.329388+0000","last_fullsized":"2024-09-21T02:06:45.329388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:01.789390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'168","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.937136+0000","last_change":"2024-09-21T02:03:58.074358+0000","last_active":"2024-09-21T02:07:15.937136+0000","last_peered":"2024-09-21T02:07:15.937136+0000","last_clean":"2024-09-21T02:07:15.937136+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:07:15.937136+0000","last_undegraded":"2024-09-21T02:07:15.937136+0000","last_fullsized":"2024-09-21T02:07:15.937136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:49.309025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'2018","reported_seq":2038,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:33.101159+0000","last_change":"2024-09-21T02:03:57.077716+0000","last_active":"2024-09-21T02:06:33.101159+0000","last_peered":"2024-09-21T02:06:33.101159+0000","last_clean":"2024-09-21T02:06:33.101159+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:06:33.101159+0000","last_undegraded":"2024-09-21T02:06:33.101159+0000","last_fullsized":"2024-09-21T02:06:33.101159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2018,"log_dups_size":0,"ondisk_log_size":2018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:15:32.970155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2018,"num_write_kb":6033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'176","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.927622+0000","last_change":"2024-09-21T02:04:45.213511+0000","last_active":"2024-09-21T02:07:15.927622+0000","last_peered":"2024-09-21T02:07:15.927622+0000","last_clean":"2024-09-21T02:07:15.927622+0000","last_became_active":"2024-09-21T02:04:45.213182+0000","last_became_peered":"2024-09-21T02:04:45.213182+0000","last_unstale":"2024-09-21T02:07:15.927622+0000","last_undegraded":"2024-09-21T02:07:15.927622+0000","last_fullsized":"2024-09-21T02:07:15.927622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:11.613618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280485+0000","last_change":"2024-09-21T02:03:57.072154+0000","last_active":"2024-09-21T02:04:45.280485+0000","last_peered":"2024-09-21T02:04:45.280485+0000","last_clean":"2024-09-21T02:04:45.280485+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:04:45.280485+0000","last_undegraded":"2024-09-21T02:04:45.280485+0000","last_fullsized":"2024-09-21T02:04:45.280485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:53.233009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.927035+0000","last_change":"2024-09-21T02:03:58.079073+0000","last_active":"2024-09-21T02:07:15.927035+0000","last_peered":"2024-09-21T02:07:15.927035+0000","last_clean":"2024-09-21T02:07:15.927035+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:07:15.927035+0000","last_undegraded":"2024-09-21T02:07:15.927035+0000","last_fullsized":"2024-09-21T02:07:15.927035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:51.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205637+0000","last_change":"2024-09-21T02:03:57.074085+0000","last_active":"2024-09-21T02:04:45.205637+0000","last_peered":"2024-09-21T02:04:45.205637+0000","last_clean":"2024-09-21T02:04:45.205637+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:04:45.205637+0000","last_undegraded":"2024-09-21T02:04:45.205637+0000","last_fullsized":"2024-09-21T02:04:45.205637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:19.393034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'201","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.937095+0000","last_change":"2024-09-21T02:03:58.087853+0000","last_active":"2024-09-21T02:07:15.937095+0000","last_peered":"2024-09-21T02:07:15.937095+0000","last_clean":"2024-09-21T02:07:15.937095+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:07:15.937095+0000","last_undegraded":"2024-09-21T02:07:15.937095+0000","last_fullsized":"2024-09-21T02:07:15.937095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:34.427166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:19.585188+0000","last_change":"2024-09-21T02:03:57.077003+0000","last_active":"2024-09-21T02:06:19.585188+0000","last_peered":"2024-09-21T02:06:19.585188+0000","last_clean":"2024-09-21T02:06:19.585188+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:06:19.585188+0000","last_undegraded":"2024-09-21T02:06:19.585188+0000","last_fullsized":"2024-09-21T02:06:19.585188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:27.589241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":985,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'197","reported_seq":215,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.909309+0000","last_change":"2024-09-21T02:03:58.084067+0000","last_active":"2024-09-21T02:07:15.909309+0000","last_peered":"2024-09-21T02:07:15.909309+0000","last_clean":"2024-09-21T02:07:15.909309+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:07:15.909309+0000","last_undegraded":"2024-09-21T02:07:15.909309+0000","last_fullsized":"2024-09-21T02:07:15.909309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:34.525699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'16","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:16.689517+0000","last_change":"2024-09-21T02:03:57.078699+0000","last_active":"2024-09-21T02:06:16.689517+0000","last_peered":"2024-09-21T02:06:16.689517+0000","last_clean":"2024-09-21T02:06:16.689517+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:06:16.689517+0000","last_undegraded":"2024-09-21T02:06:16.689517+0000","last_fullsized":"2024-09-21T02:06:16.689517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:36.628420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'215","reported_seq":233,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936059+0000","last_change":"2024-09-21T02:03:58.084524+0000","last_active":"2024-09-21T02:07:15.936059+0000","last_peered":"2024-09-21T02:07:15.936059+0000","last_clean":"2024-09-21T02:07:15.936059+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:07:15.936059+0000","last_undegraded":"2024-09-21T02:07:15.936059+0000","last_fullsized":"2024-09-21T02:07:15.936059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:06.244858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:11.588175+0000","last_change":"2024-09-21T02:03:57.067063+0000","last_active":"2024-09-21T02:06:11.588175+0000","last_peered":"2024-09-21T02:06:11.588175+0000","last_clean":"2024-09-21T02:06:11.588175+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:06:11.588175+0000","last_undegraded":"2024-09-21T02:06:11.588175+0000","last_fullsized":"2024-09-21T02:06:11.588175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:16.677652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.933136+0000","last_change":"2024-09-21T02:03:58.079088+0000","last_active":"2024-09-21T02:07:15.933136+0000","last_peered":"2024-09-21T02:07:15.933136+0000","last_clean":"2024-09-21T02:07:15.933136+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:07:15.933136+0000","last_undegraded":"2024-09-21T02:07:15.933136+0000","last_fullsized":"2024-09-21T02:07:15.933136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:59.839794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208099+0000","last_change":"2024-09-21T02:03:57.072242+0000","last_active":"2024-09-21T02:04:45.208099+0000","last_peered":"2024-09-21T02:04:45.208099+0000","last_clean":"2024-09-21T02:04:45.208099+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:04:45.208099+0000","last_undegraded":"2024-09-21T02:04:45.208099+0000","last_fullsized":"2024-09-21T02:04:45.208099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:52.775207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936043+0000","last_change":"2024-09-21T02:03:58.081720+0000","last_active":"2024-09-21T02:07:15.936043+0000","last_peered":"2024-09-21T02:07:15.936043+0000","last_clean":"2024-09-21T02:07:15.936043+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:07:15.936043+0000","last_undegraded":"2024-09-21T02:07:15.936043+0000","last_fullsized":"2024-09-21T02:07:15.936043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:58.519960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.208098+0000","last_change":"2024-09-21T02:03:57.076399+0000","last_active":"2024-09-21T02:04:45.208098+0000","last_peered":"2024-09-21T02:04:45.208098+0000","last_clean":"2024-09-21T02:04:45.208098+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:04:45.208098+0000","last_undegraded":"2024-09-21T02:04:45.208098+0000","last_fullsized":"2024-09-21T02:04:45.208098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:34.415652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'165","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932552+0000","last_change":"2024-09-21T02:03:58.080200+0000","last_active":"2024-09-21T02:07:15.932552+0000","last_peered":"2024-09-21T02:07:15.932552+0000","last_clean":"2024-09-21T02:07:15.932552+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:07:15.932552+0000","last_undegraded":"2024-09-21T02:07:15.932552+0000","last_fullsized":"2024-09-21T02:07:15.932552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:07.607556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207953+0000","last_change":"2024-09-21T02:03:57.074738+0000","last_active":"2024-09-21T02:04:45.207953+0000","last_peered":"2024-09-21T02:04:45.207953+0000","last_clean":"2024-09-21T02:04:45.207953+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:04:45.207953+0000","last_undegraded":"2024-09-21T02:04:45.207953+0000","last_fullsized":"2024-09-21T02:04:45.207953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:11.528618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'175","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936586+0000","last_change":"2024-09-21T02:03:58.087974+0000","last_active":"2024-09-21T02:07:15.936586+0000","last_peered":"2024-09-21T02:07:15.936586+0000","last_clean":"2024-09-21T02:07:15.936586+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:07:15.936586+0000","last_undegraded":"2024-09-21T02:07:15.936586+0000","last_fullsized":"2024-09-21T02:07:15.936586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:31.970527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211129+0000","last_change":"2024-09-21T02:03:47.428415+0000","last_active":"2024-09-21T02:04:45.211129+0000","last_peered":"2024-09-21T02:04:45.211129+0000","last_clean":"2024-09-21T02:04:45.211129+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:04:45.211129+0000","last_undegraded":"2024-09-21T02:04:45.211129+0000","last_fullsized":"2024-09-21T02:04:45.211129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:45.801953+0000","last_clean_scrub_stamp":"2024-09-21T02:03:45.801953+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:27.544615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205332+0000","last_change":"2024-09-21T02:03:57.072718+0000","last_active":"2024-09-21T02:04:45.205332+0000","last_peered":"2024-09-21T02:04:45.205332+0000","last_clean":"2024-09-21T02:04:45.205332+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:04:45.205332+0000","last_undegraded":"2024-09-21T02:04:45.205332+0000","last_fullsized":"2024-09-21T02:04:45.205332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:29.113238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'183","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932225+0000","last_change":"2024-09-21T02:03:58.081533+0000","last_active":"2024-09-21T02:07:15.932225+0000","last_peered":"2024-09-21T02:07:15.932225+0000","last_clean":"2024-09-21T02:07:15.932225+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:07:15.932225+0000","last_undegraded":"2024-09-21T02:07:15.932225+0000","last_fullsized":"2024-09-21T02:07:15.932225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:38.655686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210923+0000","last_change":"2024-09-21T02:03:57.080937+0000","last_active":"2024-09-21T02:04:45.210923+0000","last_peered":"2024-09-21T02:04:45.210923+0000","last_clean":"2024-09-21T02:04:45.210923+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:04:45.210923+0000","last_undegraded":"2024-09-21T02:04:45.210923+0000","last_fullsized":"2024-09-21T02:04:45.210923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:28.620187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'165","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:20.332299+0000","last_change":"2024-09-21T02:07:20.332299+0000","last_active":"2024-09-21T02:07:20.332299+0000","last_peered":"2024-09-21T02:07:20.332299+0000","last_clean":"2024-09-21T02:07:20.332299+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:07:20.332299+0000","last_undegraded":"2024-09-21T02:07:20.332299+0000","last_fullsized":"2024-09-21T02:07:20.332299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-09-21T02:07:20.332215+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-09-21T02:07:20.332215+0000","last_clean_scrub_stamp":"2024-09-21T02:07:20.332215+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:53.857052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:47.179470+0000","last_change":"2024-09-21T02:03:57.075241+0000","last_active":"2024-09-21T02:06:47.179470+0000","last_peered":"2024-09-21T02:06:47.179470+0000","last_clean":"2024-09-21T02:06:47.179470+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:06:47.179470+0000","last_undegraded":"2024-09-21T02:06:47.179470+0000","last_fullsized":"2024-09-21T02:06:47.179470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:15:23.892410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":595,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'165","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.899103+0000","last_change":"2024-09-21T02:03:58.077466+0000","last_active":"2024-09-21T02:07:15.899103+0000","last_peered":"2024-09-21T02:07:15.899103+0000","last_clean":"2024-09-21T02:07:15.899103+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:07:15.899103+0000","last_undegraded":"2024-09-21T02:07:15.899103+0000","last_fullsized":"2024-09-21T02:07:15.899103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:47.835962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.205672+0000","last_change":"2024-09-21T02:03:57.078173+0000","last_active":"2024-09-21T02:04:45.205672+0000","last_peered":"2024-09-21T02:04:45.205672+0000","last_clean":"2024-09-21T02:04:45.205672+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:04:45.205672+0000","last_undegraded":"2024-09-21T02:04:45.205672+0000","last_fullsized":"2024-09-21T02:04:45.205672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:04.884509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'177","reported_seq":195,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.923138+0000","last_change":"2024-09-21T02:03:58.082918+0000","last_active":"2024-09-21T02:07:15.923138+0000","last_peered":"2024-09-21T02:07:15.923138+0000","last_clean":"2024-09-21T02:07:15.923138+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:07:15.923138+0000","last_undegraded":"2024-09-21T02:07:15.923138+0000","last_fullsized":"2024-09-21T02:07:15.923138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:56.853778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207553+0000","last_change":"2024-09-21T02:03:57.081136+0000","last_active":"2024-09-21T02:04:45.207553+0000","last_peered":"2024-09-21T02:04:45.207553+0000","last_clean":"2024-09-21T02:04:45.207553+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:04:45.207553+0000","last_undegraded":"2024-09-21T02:04:45.207553+0000","last_fullsized":"2024-09-21T02:04:45.207553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:30.883565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934011+0000","last_change":"2024-09-21T02:03:58.078616+0000","last_active":"2024-09-21T02:07:15.934011+0000","last_peered":"2024-09-21T02:07:15.934011+0000","last_clean":"2024-09-21T02:07:15.934011+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:07:15.934011+0000","last_undegraded":"2024-09-21T02:07:15.934011+0000","last_fullsized":"2024-09-21T02:07:15.934011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:33.326771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211858+0000","last_change":"2024-09-21T02:03:57.081422+0000","last_active":"2024-09-21T02:04:45.211858+0000","last_peered":"2024-09-21T02:04:45.211858+0000","last_clean":"2024-09-21T02:04:45.211858+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:04:45.211858+0000","last_undegraded":"2024-09-21T02:04:45.211858+0000","last_fullsized":"2024-09-21T02:04:45.211858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.961887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.899561+0000","last_change":"2024-09-21T02:03:58.077511+0000","last_active":"2024-09-21T02:07:15.899561+0000","last_peered":"2024-09-21T02:07:15.899561+0000","last_clean":"2024-09-21T02:07:15.899561+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:07:15.899561+0000","last_undegraded":"2024-09-21T02:07:15.899561+0000","last_fullsized":"2024-09-21T02:07:15.899561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:39.848588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'22","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:48.123518+0000","last_change":"2024-09-21T02:03:57.079386+0000","last_active":"2024-09-21T02:06:48.123518+0000","last_peered":"2024-09-21T02:06:48.123518+0000","last_clean":"2024-09-21T02:06:48.123518+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:06:48.123518+0000","last_undegraded":"2024-09-21T02:06:48.123518+0000","last_fullsized":"2024-09-21T02:06:48.123518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:29.054239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'166","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.936423+0000","last_change":"2024-09-21T02:03:58.074910+0000","last_active":"2024-09-21T02:07:15.936423+0000","last_peered":"2024-09-21T02:07:15.936423+0000","last_clean":"2024-09-21T02:07:15.936423+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:07:15.936423+0000","last_undegraded":"2024-09-21T02:07:15.936423+0000","last_fullsized":"2024-09-21T02:07:15.936423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:16.193808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:09.584145+0000","last_change":"2024-09-21T02:03:57.080088+0000","last_active":"2024-09-21T02:06:09.584145+0000","last_peered":"2024-09-21T02:06:09.584145+0000","last_clean":"2024-09-21T02:06:09.584145+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:06:09.584145+0000","last_undegraded":"2024-09-21T02:06:09.584145+0000","last_fullsized":"2024-09-21T02:06:09.584145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'183","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.914296+0000","last_change":"2024-09-21T02:03:58.077974+0000","last_active":"2024-09-21T02:07:15.914296+0000","last_peered":"2024-09-21T02:07:15.914296+0000","last_clean":"2024-09-21T02:07:15.914296+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:07:15.914296+0000","last_undegraded":"2024-09-21T02:07:15.914296+0000","last_fullsized":"2024-09-21T02:07:15.914296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:10.821810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.210942+0000","last_change":"2024-09-21T02:03:57.082347+0000","last_active":"2024-09-21T02:04:45.210942+0000","last_peered":"2024-09-21T02:04:45.210942+0000","last_clean":"2024-09-21T02:04:45.210942+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:04:45.210942+0000","last_undegraded":"2024-09-21T02:04:45.210942+0000","last_fullsized":"2024-09-21T02:04:45.210942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:17.949684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.909177+0000","last_change":"2024-09-21T02:03:58.083459+0000","last_active":"2024-09-21T02:07:15.909177+0000","last_peered":"2024-09-21T02:07:15.909177+0000","last_clean":"2024-09-21T02:07:15.909177+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:07:15.909177+0000","last_undegraded":"2024-09-21T02:07:15.909177+0000","last_fullsized":"2024-09-21T02:07:15.909177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.643506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207751+0000","last_change":"2024-09-21T02:03:57.078574+0000","last_active":"2024-09-21T02:04:45.207751+0000","last_peered":"2024-09-21T02:04:45.207751+0000","last_clean":"2024-09-21T02:04:45.207751+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:04:45.207751+0000","last_undegraded":"2024-09-21T02:04:45.207751+0000","last_fullsized":"2024-09-21T02:04:45.207751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:56.648755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'173","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.909293+0000","last_change":"2024-09-21T02:03:58.080840+0000","last_active":"2024-09-21T02:07:15.909293+0000","last_peered":"2024-09-21T02:07:15.909293+0000","last_clean":"2024-09-21T02:07:15.909293+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:07:15.909293+0000","last_undegraded":"2024-09-21T02:07:15.909293+0000","last_fullsized":"2024-09-21T02:07:15.909293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.484549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.207926+0000","last_change":"2024-09-21T02:03:57.079646+0000","last_active":"2024-09-21T02:04:45.207926+0000","last_peered":"2024-09-21T02:04:45.207926+0000","last_clean":"2024-09-21T02:04:45.207926+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:04:45.207926+0000","last_undegraded":"2024-09-21T02:04:45.207926+0000","last_fullsized":"2024-09-21T02:04:45.207926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:44.654000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'164","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.917872+0000","last_change":"2024-09-21T02:03:58.083361+0000","last_active":"2024-09-21T02:07:15.917872+0000","last_peered":"2024-09-21T02:07:15.917872+0000","last_clean":"2024-09-21T02:07:15.917872+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:07:15.917872+0000","last_undegraded":"2024-09-21T02:07:15.917872+0000","last_fullsized":"2024-09-21T02:07:15.917872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:54.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:05:47.141927+0000","last_change":"2024-09-21T02:03:57.084293+0000","last_active":"2024-09-21T02:05:47.141927+0000","last_peered":"2024-09-21T02:05:47.141927+0000","last_clean":"2024-09-21T02:05:47.141927+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:05:47.141927+0000","last_undegraded":"2024-09-21T02:05:47.141927+0000","last_fullsized":"2024-09-21T02:05:47.141927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:29.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'154","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.932692+0000","last_change":"2024-09-21T02:03:58.079884+0000","last_active":"2024-09-21T02:07:15.932692+0000","last_peered":"2024-09-21T02:07:15.932692+0000","last_clean":"2024-09-21T02:07:15.932692+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:07:15.932692+0000","last_undegraded":"2024-09-21T02:07:15.932692+0000","last_fullsized":"2024-09-21T02:07:15.932692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:01.561599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'205","reported_seq":223,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.930285+0000","last_change":"2024-09-21T02:03:58.067172+0000","last_active":"2024-09-21T02:07:15.930285+0000","last_peered":"2024-09-21T02:07:15.930285+0000","last_clean":"2024-09-21T02:07:15.930285+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:07:15.930285+0000","last_undegraded":"2024-09-21T02:07:15.930285+0000","last_fullsized":"2024-09-21T02:07:15.930285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:21.359735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.211388+0000","last_change":"2024-09-21T02:03:57.081840+0000","last_active":"2024-09-21T02:04:45.211388+0000","last_peered":"2024-09-21T02:04:45.211388+0000","last_clean":"2024-09-21T02:04:45.211388+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:04:45.211388+0000","last_undegraded":"2024-09-21T02:04:45.211388+0000","last_fullsized":"2024-09-21T02:04:45.211388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:52.765342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'162","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.934153+0000","last_change":"2024-09-21T02:03:58.081917+0000","last_active":"2024-09-21T02:07:15.934153+0000","last_peered":"2024-09-21T02:07:15.934153+0000","last_clean":"2024-09-21T02:07:15.934153+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:07:15.934153+0000","last_undegraded":"2024-09-21T02:07:15.934153+0000","last_fullsized":"2024-09-21T02:07:15.934153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:05.285389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:06:08.902449+0000","last_change":"2024-09-21T02:03:57.082892+0000","last_active":"2024-09-21T02:06:08.902449+0000","last_peered":"2024-09-21T02:06:08.902449+0000","last_clean":"2024-09-21T02:06:08.902449+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:06:08.902449+0000","last_undegraded":"2024-09-21T02:06:08.902449+0000","last_fullsized":"2024-09-21T02:06:08.902449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:44.642309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'160","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.924121+0000","last_change":"2024-09-21T02:03:58.080392+0000","last_active":"2024-09-21T02:07:15.924121+0000","last_peered":"2024-09-21T02:07:15.924121+0000","last_clean":"2024-09-21T02:07:15.924121+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:07:15.924121+0000","last_undegraded":"2024-09-21T02:07:15.924121+0000","last_fullsized":"2024-09-21T02:07:15.924121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:16.560793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:04:45.280872+0000","last_change":"2024-09-21T02:03:57.080049+0000","last_active":"2024-09-21T02:04:45.280872+0000","last_peered":"2024-09-21T02:04:45.280872+0000","last_clean":"2024-09-21T02:04:45.280872+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:04:45.280872+0000","last_undegraded":"2024-09-21T02:04:45.280872+0000","last_fullsized":"2024-09-21T02:04:45.280872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:56.045249+0000","last_clean_scrub_stamp":"2024-09-21T02:03:56.045249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:20.357044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'181","reported_seq":199,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:15.927298+0000","last_change":"2024-09-21T02:03:58.079384+0000","last_active":"2024-09-21T02:07:15.927298+0000","last_peered":"2024-09-21T02:07:15.927298+0000","last_clean":"2024-09-21T02:07:15.927298+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:07:15.927298+0000","last_undegraded":"2024-09-21T02:07:15.927298+0000","last_fullsized":"2024-09-21T02:07:15.927298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:03:57.050799+0000","last_clean_scrub_stamp":"2024-09-21T02:03:57.050799+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.862102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'13","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:19.285919+0000","last_change":"2024-09-21T02:07:19.285919+0000","last_active":"2024-09-21T02:07:19.285919+0000","last_peered":"2024-09-21T02:07:19.285919+0000","last_clean":"2024-09-21T02:07:19.285919+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:07:19.285919+0000","last_undegraded":"2024-09-21T02:07:19.285919+0000","last_fullsized":"2024-09-21T02:07:19.285919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13","last_scrub_stamp":"2024-09-21T02:07:19.285838+0000","last_deep_scrub":"23'13","last_deep_scrub_stamp":"2024-09-21T02:07:19.285838+0000","last_clean_scrub_stamp":"2024-09-21T02:07:19.285838+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:00.264480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11255,"ondisk_log_size":11255,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7623955,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3826,"num_write_kb":11018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15409152,"data_stored":15247834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3804,"ondisk_log_size":3804,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57564,"kb_used_data":4172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314276,"statfs":{"total":96636764160,"available":96577818624,"internally_reserved":0,"allocated":4272128,"data_stored":4100839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":86315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":86048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83232,"kb_used_data":7696,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94288608,"statfs":{"total":96636764160,"available":96551534592,"internally_reserved":0,"allocated":7880704,"data_stored":7705597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":86260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":85881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53572,"kb_used_data":4340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318268,"statfs":{"total":96636764160,"available":96581906432,"internally_reserved":0,"allocated":4444160,"data_stored":4280988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":843776,"data_stored":675965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7647232,"data_stored":7620660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4196051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3448832,"data_stored":3425707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:07:22.632 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.632 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.632 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.632 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.632 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.632 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.633 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.634 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.635 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.636 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.637 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.638 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T02:03:45.801953+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.639 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.640 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-21T02:03:56.045249+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=56, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-21T02:03:57.050799+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=3, tm_sec=57, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=2, tm_min=7, tm_sec=17, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T02:07:22.641 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T02:07:28.416 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:07:28.419 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:07:42.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:07:42.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.769+0000 7fc053e20640 1 -- 172.21.15.88:0/2017864292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc034003a00 msgr2=0x7fc034003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:42.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.769+0000 7fc053e20640 1 --2- 172.21.15.88:0/2017864292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc034003a00 0x7fc034003e70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009d00 tx=0x7fc034005830 comp rx=0 tx=0).stop 2024-09-21T02:07:42.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.769+0000 7fc0523bd640 1 -- 172.21.15.88:0/2017864292 reap_dead start 2024-09-21T02:07:42.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.769+0000 7fc053e20640 1 -- 172.21.15.88:0/2017864292 shutdown_connections 2024-09-21T02:07:42.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.769+0000 7fc053e20640 1 -- 172.21.15.88:0/2017864292 >> 172.21.15.88:0/2017864292 conn(0x7fc04c0729d0 msgr2=0x7fc04c072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:42.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.769+0000 7fc053e20640 1 -- 172.21.15.88:0/2017864292 shutdown_connections 2024-09-21T02:07:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.769+0000 7fc053e20640 1 -- 172.21.15.88:0/2017864292 wait complete. 2024-09-21T02:07:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.770+0000 7fc053e20640 1 Processor -- start 2024-09-21T02:07:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.770+0000 7fc053e20640 1 -- start start 2024-09-21T02:07:42.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc053e20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc034003a00 0x7fc04c14a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:42.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc053e20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc04c157580 0x7fc04c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:42.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc053e20640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc04c149eb0 0x7fc04c149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:42.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc053e20640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c15a5b0 con 0x7fc04c149eb0 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc053e20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c070ab0 con 0x7fc034003a00 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc053e20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c0c8f30 con 0x7fc04c157580 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc04c149eb0 0x7fc04c149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc04c149eb0 0x7fc04c149430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48192/0 (socket says 172.21.15.88:48192) 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 -- 172.21.15.88:0/1459523142 learned_addr learned my addr 172.21.15.88:0/1459523142 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc051bbc640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc034003a00 0x7fc04c14a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0513bb640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc04c157580 0x7fc04c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 -- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc04c157580 msgr2=0x7fc04c148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc04c157580 0x7fc04c148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 -- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc034003a00 msgr2=0x7fc04c14a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc034003a00 0x7fc04c14a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 -- 172.21.15.88:0/1459523142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c157b90 con 0x7fc04c149eb0 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0513bb640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc04c157580 0x7fc04c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc0523bd640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc04c149eb0 0x7fc04c149430 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc04c15dc40 tx=0x7fc044011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc04a7fc640 1 -- 172.21.15.88:0/1459523142 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04400fd40 con 0x7fc04c149eb0 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.772+0000 7fc04a7fc640 1 -- 172.21.15.88:0/1459523142 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc044004b20 con 0x7fc04c149eb0 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.773+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c1570a0 con 0x7fc04c149eb0 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.773+0000 7fc04a7fc640 1 -- 172.21.15.88:0/1459523142 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044003970 con 0x7fc04c149eb0 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.773+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c157e30 con 0x7fc04c149eb0 2024-09-21T02:07:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.773+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c072360 con 0x7fc04c149eb0 2024-09-21T02:07:42.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.774+0000 7fc04a7fc640 1 -- 172.21.15.88:0/1459523142 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc044020020 con 0x7fc04c149eb0 2024-09-21T02:07:42.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.775+0000 7fc04a7fc640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc038078b90 0x7fc03807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:42.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.775+0000 7fc04a7fc640 1 -- 172.21.15.88:0/1459523142 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc04409b610 con 0x7fc04c149eb0 2024-09-21T02:07:42.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.775+0000 7fc051bbc640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc038078b90 0x7fc03807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:42.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.776+0000 7fc051bbc640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc038078b90 0x7fc03807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc03c035000 tx=0x7fc03c0046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:42.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.779+0000 7fc04a7fc640 1 -- 172.21.15.88:0/1459523142 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc044063a60 con 0x7fc04c149eb0 2024-09-21T02:07:42.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.953+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 --> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc04c151b60 con 0x7fc038078b90 2024-09-21T02:07:42.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.958+0000 7fc04a7fc640 1 -- 172.21.15.88:0/1459523142 <== mgr.4117 v2:172.21.15.195:6800/4016634266 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253465 (secure 0 0 0) 0x7fc04c151b60 con 0x7fc038078b90 2024-09-21T02:07:42.961 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:42.962 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T02:07:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc038078b90 msgr2=0x7fc03807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc038078b90 0x7fc03807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc03c035000 tx=0x7fc03c0046c0 comp rx=0 tx=0).stop 2024-09-21T02:07:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc04c149eb0 msgr2=0x7fc04c149430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc04c149eb0 0x7fc04c149430 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc04c15dc40 tx=0x7fc044011f90 comp rx=0 tx=0).stop 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 shutdown_connections 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7fc038078b90 0x7fc03807b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc04c149eb0 0x7fc04c149430 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc04c157580 0x7fc04c148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 --2- 172.21.15.88:0/1459523142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc034003a00 0x7fc04c14a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 >> 172.21.15.88:0/1459523142 conn(0x7fc04c0729d0 msgr2=0x7fc04c0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 shutdown_connections 2024-09-21T02:07:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:42.961+0000 7fc053e20640 1 -- 172.21.15.88:0/1459523142 wait complete. 2024-09-21T02:07:42.973 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-21T02:07:42.881781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8214323,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":71,"num_read_kb":60,"num_write":13143,"num_write_kb":25178470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15185,"ondisk_log_size":15185,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347952,"kb_used_data":18032,"kb_used_omap":12,"kb_used_meta":329779,"kb_avail":754626768,"statfs":{"total":773094113280,"available":772737810432,"internally_reserved":0,"allocated":18464768,"data_stored":17108142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":337694258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"3.3f","version":"23'167","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:38.709887+0000","last_change":"2024-09-21T02:07:38.709887+0000","last_active":"2024-09-21T02:07:38.709887+0000","last_peered":"2024-09-21T02:07:38.709887+0000","last_clean":"2024-09-21T02:07:38.709887+0000","last_became_active":"2024-09-21T02:03:58.086927+0000","last_became_peered":"2024-09-21T02:03:58.086927+0000","last_unstale":"2024-09-21T02:07:38.709887+0000","last_undegraded":"2024-09-21T02:07:38.709887+0000","last_fullsized":"2024-09-21T02:07:38.709887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'167","last_scrub_stamp":"2024-09-21T02:07:38.709757+0000","last_deep_scrub":"23'167","last_deep_scrub_stamp":"2024-09-21T02:07:38.709757+0000","last_clean_scrub_stamp":"2024-09-21T02:07:38.709757+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:23.068633+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'152","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:32.115367+0000","last_change":"2024-09-21T02:07:32.115367+0000","last_active":"2024-09-21T02:07:32.115367+0000","last_peered":"2024-09-21T02:07:32.115367+0000","last_clean":"2024-09-21T02:07:32.115367+0000","last_became_active":"2024-09-21T02:03:58.082502+0000","last_became_peered":"2024-09-21T02:03:58.082502+0000","last_unstale":"2024-09-21T02:07:32.115367+0000","last_undegraded":"2024-09-21T02:07:32.115367+0000","last_fullsized":"2024-09-21T02:07:32.115367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'152","last_scrub_stamp":"2024-09-21T02:07:32.115267+0000","last_deep_scrub":"23'152","last_deep_scrub_stamp":"2024-09-21T02:07:32.115267+0000","last_clean_scrub_stamp":"2024-09-21T02:07:32.115267+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:04.829779+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'178","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:28.368710+0000","last_change":"2024-09-21T02:07:28.368710+0000","last_active":"2024-09-21T02:07:28.368710+0000","last_peered":"2024-09-21T02:07:28.368710+0000","last_clean":"2024-09-21T02:07:28.368710+0000","last_became_active":"2024-09-21T02:03:58.076753+0000","last_became_peered":"2024-09-21T02:03:58.076753+0000","last_unstale":"2024-09-21T02:07:28.368710+0000","last_undegraded":"2024-09-21T02:07:28.368710+0000","last_fullsized":"2024-09-21T02:07:28.368710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'178","last_scrub_stamp":"2024-09-21T02:07:28.368611+0000","last_deep_scrub":"23'178","last_deep_scrub_stamp":"2024-09-21T02:07:28.368611+0000","last_clean_scrub_stamp":"2024-09-21T02:07:28.368611+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:15.293708+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'193","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:34.349539+0000","last_change":"2024-09-21T02:07:34.349539+0000","last_active":"2024-09-21T02:07:34.349539+0000","last_peered":"2024-09-21T02:07:34.349539+0000","last_clean":"2024-09-21T02:07:34.349539+0000","last_became_active":"2024-09-21T02:03:58.079995+0000","last_became_peered":"2024-09-21T02:03:58.079995+0000","last_unstale":"2024-09-21T02:07:34.349539+0000","last_undegraded":"2024-09-21T02:07:34.349539+0000","last_fullsized":"2024-09-21T02:07:34.349539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'193","last_scrub_stamp":"2024-09-21T02:07:34.349418+0000","last_deep_scrub":"23'193","last_deep_scrub_stamp":"2024-09-21T02:07:34.349418+0000","last_clean_scrub_stamp":"2024-09-21T02:07:34.349418+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:13.004678+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'174","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:39.719702+0000","last_change":"2024-09-21T02:07:39.719702+0000","last_active":"2024-09-21T02:07:39.719702+0000","last_peered":"2024-09-21T02:07:39.719702+0000","last_clean":"2024-09-21T02:07:39.719702+0000","last_became_active":"2024-09-21T02:03:58.085406+0000","last_became_peered":"2024-09-21T02:03:58.085406+0000","last_unstale":"2024-09-21T02:07:39.719702+0000","last_undegraded":"2024-09-21T02:07:39.719702+0000","last_fullsized":"2024-09-21T02:07:39.719702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'174","last_scrub_stamp":"2024-09-21T02:07:39.719593+0000","last_deep_scrub":"23'174","last_deep_scrub_stamp":"2024-09-21T02:07:39.719593+0000","last_clean_scrub_stamp":"2024-09-21T02:07:39.719593+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:02.948575+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:33.334254+0000","last_change":"2024-09-21T02:07:33.334254+0000","last_active":"2024-09-21T02:07:33.334254+0000","last_peered":"2024-09-21T02:07:33.334254+0000","last_clean":"2024-09-21T02:07:33.334254+0000","last_became_active":"2024-09-21T02:03:58.082321+0000","last_became_peered":"2024-09-21T02:03:58.082321+0000","last_unstale":"2024-09-21T02:07:33.334254+0000","last_undegraded":"2024-09-21T02:07:33.334254+0000","last_fullsized":"2024-09-21T02:07:33.334254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-09-21T02:07:33.334149+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-09-21T02:07:33.334149+0000","last_clean_scrub_stamp":"2024-09-21T02:07:33.334149+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:24.013602+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'188","reported_seq":214,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:37.693372+0000","last_change":"2024-09-21T02:07:37.693372+0000","last_active":"2024-09-21T02:07:37.693372+0000","last_peered":"2024-09-21T02:07:37.693372+0000","last_clean":"2024-09-21T02:07:37.693372+0000","last_became_active":"2024-09-21T02:03:58.085615+0000","last_became_peered":"2024-09-21T02:03:58.085615+0000","last_unstale":"2024-09-21T02:07:37.693372+0000","last_undegraded":"2024-09-21T02:07:37.693372+0000","last_fullsized":"2024-09-21T02:07:37.693372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'188","last_scrub_stamp":"2024-09-21T02:07:37.693264+0000","last_deep_scrub":"23'188","last_deep_scrub_stamp":"2024-09-21T02:07:37.693264+0000","last_clean_scrub_stamp":"2024-09-21T02:07:37.693264+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:09.403204+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'204","reported_seq":230,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:33.466460+0000","last_change":"2024-09-21T02:07:33.466460+0000","last_active":"2024-09-21T02:07:33.466460+0000","last_peered":"2024-09-21T02:07:33.466460+0000","last_clean":"2024-09-21T02:07:33.466460+0000","last_became_active":"2024-09-21T02:03:58.081443+0000","last_became_peered":"2024-09-21T02:03:58.081443+0000","last_unstale":"2024-09-21T02:07:33.466460+0000","last_undegraded":"2024-09-21T02:07:33.466460+0000","last_fullsized":"2024-09-21T02:07:33.466460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'204","last_scrub_stamp":"2024-09-21T02:07:33.466360+0000","last_deep_scrub":"23'204","last_deep_scrub_stamp":"2024-09-21T02:07:33.466360+0000","last_clean_scrub_stamp":"2024-09-21T02:07:33.466360+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:56.324101+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'178","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:36.724170+0000","last_change":"2024-09-21T02:07:36.724170+0000","last_active":"2024-09-21T02:07:36.724170+0000","last_peered":"2024-09-21T02:07:36.724170+0000","last_clean":"2024-09-21T02:07:36.724170+0000","last_became_active":"2024-09-21T02:03:58.087595+0000","last_became_peered":"2024-09-21T02:03:58.087595+0000","last_unstale":"2024-09-21T02:07:36.724170+0000","last_undegraded":"2024-09-21T02:07:36.724170+0000","last_fullsized":"2024-09-21T02:07:36.724170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'178","last_scrub_stamp":"2024-09-21T02:07:36.724058+0000","last_deep_scrub":"23'178","last_deep_scrub_stamp":"2024-09-21T02:07:36.724058+0000","last_clean_scrub_stamp":"2024-09-21T02:07:36.724058+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:09.533219+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'190","reported_seq":216,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:33.270119+0000","last_change":"2024-09-21T02:07:33.270119+0000","last_active":"2024-09-21T02:07:33.270119+0000","last_peered":"2024-09-21T02:07:33.270119+0000","last_clean":"2024-09-21T02:07:33.270119+0000","last_became_active":"2024-09-21T02:03:58.080745+0000","last_became_peered":"2024-09-21T02:03:58.080745+0000","last_unstale":"2024-09-21T02:07:33.270119+0000","last_undegraded":"2024-09-21T02:07:33.270119+0000","last_fullsized":"2024-09-21T02:07:33.270119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'190","last_scrub_stamp":"2024-09-21T02:07:33.270028+0000","last_deep_scrub":"23'190","last_deep_scrub_stamp":"2024-09-21T02:07:33.270028+0000","last_clean_scrub_stamp":"2024-09-21T02:07:33.270028+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:00.711999+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'214","reported_seq":240,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:32.278977+0000","last_change":"2024-09-21T02:07:32.278977+0000","last_active":"2024-09-21T02:07:32.278977+0000","last_peered":"2024-09-21T02:07:32.278977+0000","last_clean":"2024-09-21T02:07:32.278977+0000","last_became_active":"2024-09-21T02:03:58.081326+0000","last_became_peered":"2024-09-21T02:03:58.081326+0000","last_unstale":"2024-09-21T02:07:32.278977+0000","last_undegraded":"2024-09-21T02:07:32.278977+0000","last_fullsized":"2024-09-21T02:07:32.278977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'214","last_scrub_stamp":"2024-09-21T02:07:32.278852+0000","last_deep_scrub":"23'214","last_deep_scrub_stamp":"2024-09-21T02:07:32.278852+0000","last_clean_scrub_stamp":"2024-09-21T02:07:32.278852+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:51.296272+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'166","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.067497+0000","last_change":"2024-09-21T02:07:26.067497+0000","last_active":"2024-09-21T02:07:26.067497+0000","last_peered":"2024-09-21T02:07:26.067497+0000","last_clean":"2024-09-21T02:07:26.067497+0000","last_became_active":"2024-09-21T02:03:58.080146+0000","last_became_peered":"2024-09-21T02:03:58.080146+0000","last_unstale":"2024-09-21T02:07:26.067497+0000","last_undegraded":"2024-09-21T02:07:26.067497+0000","last_fullsized":"2024-09-21T02:07:26.067497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'166","last_scrub_stamp":"2024-09-21T02:07:26.067391+0000","last_deep_scrub":"23'166","last_deep_scrub_stamp":"2024-09-21T02:07:26.067391+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.067391+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:40:30.373331+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'177","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:32.416721+0000","last_change":"2024-09-21T02:07:32.416721+0000","last_active":"2024-09-21T02:07:32.416721+0000","last_peered":"2024-09-21T02:07:32.416721+0000","last_clean":"2024-09-21T02:07:32.416721+0000","last_became_active":"2024-09-21T02:03:58.078866+0000","last_became_peered":"2024-09-21T02:03:58.078866+0000","last_unstale":"2024-09-21T02:07:32.416721+0000","last_undegraded":"2024-09-21T02:07:32.416721+0000","last_fullsized":"2024-09-21T02:07:32.416721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-09-21T02:07:32.416615+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-09-21T02:07:32.416615+0000","last_clean_scrub_stamp":"2024-09-21T02:07:32.416615+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:20.895556+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'172","reported_seq":198,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.318453+0000","last_change":"2024-09-21T02:07:25.318453+0000","last_active":"2024-09-21T02:07:25.318453+0000","last_peered":"2024-09-21T02:07:25.318453+0000","last_clean":"2024-09-21T02:07:25.318453+0000","last_became_active":"2024-09-21T02:03:58.073859+0000","last_became_peered":"2024-09-21T02:03:58.073859+0000","last_unstale":"2024-09-21T02:07:25.318453+0000","last_undegraded":"2024-09-21T02:07:25.318453+0000","last_fullsized":"2024-09-21T02:07:25.318453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'172","last_scrub_stamp":"2024-09-21T02:07:25.318354+0000","last_deep_scrub":"23'172","last_deep_scrub_stamp":"2024-09-21T02:07:25.318354+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.318354+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:44.496221+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'177","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:31.080288+0000","last_change":"2024-09-21T02:07:31.080288+0000","last_active":"2024-09-21T02:07:31.080288+0000","last_peered":"2024-09-21T02:07:31.080288+0000","last_clean":"2024-09-21T02:07:31.080288+0000","last_became_active":"2024-09-21T02:03:58.068443+0000","last_became_peered":"2024-09-21T02:03:58.068443+0000","last_unstale":"2024-09-21T02:07:31.080288+0000","last_undegraded":"2024-09-21T02:07:31.080288+0000","last_fullsized":"2024-09-21T02:07:31.080288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-09-21T02:07:31.080176+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-09-21T02:07:31.080176+0000","last_clean_scrub_stamp":"2024-09-21T02:07:31.080176+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:36.126865+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:32.300561+0000","last_change":"2024-09-21T02:07:32.300561+0000","last_active":"2024-09-21T02:07:32.300561+0000","last_peered":"2024-09-21T02:07:32.300561+0000","last_clean":"2024-09-21T02:07:32.300561+0000","last_became_active":"2024-09-21T02:03:58.084023+0000","last_became_peered":"2024-09-21T02:03:58.084023+0000","last_unstale":"2024-09-21T02:07:32.300561+0000","last_undegraded":"2024-09-21T02:07:32.300561+0000","last_fullsized":"2024-09-21T02:07:32.300561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-09-21T02:07:32.300468+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-09-21T02:07:32.300468+0000","last_clean_scrub_stamp":"2024-09-21T02:07:32.300468+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:20.631779+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"23'176","reported_seq":196,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:27.365240+0000","last_change":"2024-09-21T02:07:27.365240+0000","last_active":"2024-09-21T02:07:27.365240+0000","last_peered":"2024-09-21T02:07:27.365240+0000","last_clean":"2024-09-21T02:07:27.365240+0000","last_became_active":"2024-09-21T02:04:45.212611+0000","last_became_peered":"2024-09-21T02:04:45.212611+0000","last_unstale":"2024-09-21T02:07:27.365240+0000","last_undegraded":"2024-09-21T02:07:27.365240+0000","last_fullsized":"2024-09-21T02:07:27.365240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'176","last_scrub_stamp":"2024-09-21T02:07:27.365115+0000","last_deep_scrub":"23'176","last_deep_scrub_stamp":"2024-09-21T02:07:27.365115+0000","last_clean_scrub_stamp":"2024-09-21T02:07:27.365115+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:53.584591+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"23'179","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.374491+0000","last_change":"2024-09-21T02:07:26.374491+0000","last_active":"2024-09-21T02:07:26.374491+0000","last_peered":"2024-09-21T02:07:26.374491+0000","last_clean":"2024-09-21T02:07:26.374491+0000","last_became_active":"2024-09-21T02:03:58.086747+0000","last_became_peered":"2024-09-21T02:03:58.086747+0000","last_unstale":"2024-09-21T02:07:26.374491+0000","last_undegraded":"2024-09-21T02:07:26.374491+0000","last_fullsized":"2024-09-21T02:07:26.374491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'179","last_scrub_stamp":"2024-09-21T02:07:26.374362+0000","last_deep_scrub":"23'179","last_deep_scrub_stamp":"2024-09-21T02:07:26.374362+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.374362+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:11.034556+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'205","reported_seq":231,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:35.772647+0000","last_change":"2024-09-21T02:07:35.772647+0000","last_active":"2024-09-21T02:07:35.772647+0000","last_peered":"2024-09-21T02:07:35.772647+0000","last_clean":"2024-09-21T02:07:35.772647+0000","last_became_active":"2024-09-21T02:03:58.078280+0000","last_became_peered":"2024-09-21T02:03:58.078280+0000","last_unstale":"2024-09-21T02:07:35.772647+0000","last_undegraded":"2024-09-21T02:07:35.772647+0000","last_fullsized":"2024-09-21T02:07:35.772647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'205","last_scrub_stamp":"2024-09-21T02:07:35.772540+0000","last_deep_scrub":"23'205","last_deep_scrub_stamp":"2024-09-21T02:07:35.772540+0000","last_clean_scrub_stamp":"2024-09-21T02:07:35.772540+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:22.776905+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'179","reported_seq":205,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:31.248487+0000","last_change":"2024-09-21T02:07:31.248487+0000","last_active":"2024-09-21T02:07:31.248487+0000","last_peered":"2024-09-21T02:07:31.248487+0000","last_clean":"2024-09-21T02:07:31.248487+0000","last_became_active":"2024-09-21T02:03:58.088570+0000","last_became_peered":"2024-09-21T02:03:58.088570+0000","last_unstale":"2024-09-21T02:07:31.248487+0000","last_undegraded":"2024-09-21T02:07:31.248487+0000","last_fullsized":"2024-09-21T02:07:31.248487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'179","last_scrub_stamp":"2024-09-21T02:07:31.248384+0000","last_deep_scrub":"23'179","last_deep_scrub_stamp":"2024-09-21T02:07:31.248384+0000","last_clean_scrub_stamp":"2024-09-21T02:07:31.248384+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:30.431615+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'154","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:30.437729+0000","last_change":"2024-09-21T02:07:30.437729+0000","last_active":"2024-09-21T02:07:30.437729+0000","last_peered":"2024-09-21T02:07:30.437729+0000","last_clean":"2024-09-21T02:07:30.437729+0000","last_became_active":"2024-09-21T02:03:58.081734+0000","last_became_peered":"2024-09-21T02:03:58.081734+0000","last_unstale":"2024-09-21T02:07:30.437729+0000","last_undegraded":"2024-09-21T02:07:30.437729+0000","last_fullsized":"2024-09-21T02:07:30.437729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'154","last_scrub_stamp":"2024-09-21T02:07:30.437631+0000","last_deep_scrub":"23'154","last_deep_scrub_stamp":"2024-09-21T02:07:30.437631+0000","last_clean_scrub_stamp":"2024-09-21T02:07:30.437631+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:08.021428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'199","reported_seq":219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:30.029911+0000","last_change":"2024-09-21T02:07:30.029911+0000","last_active":"2024-09-21T02:07:30.029911+0000","last_peered":"2024-09-21T02:07:30.029911+0000","last_clean":"2024-09-21T02:07:30.029911+0000","last_became_active":"2024-09-21T02:04:45.212587+0000","last_became_peered":"2024-09-21T02:04:45.212587+0000","last_unstale":"2024-09-21T02:07:30.029911+0000","last_undegraded":"2024-09-21T02:07:30.029911+0000","last_fullsized":"2024-09-21T02:07:30.029911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'199","last_scrub_stamp":"2024-09-21T02:07:30.029814+0000","last_deep_scrub":"23'199","last_deep_scrub_stamp":"2024-09-21T02:07:30.029814+0000","last_clean_scrub_stamp":"2024-09-21T02:07:30.029814+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:02.965488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"23'150","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:31.404434+0000","last_change":"2024-09-21T02:07:31.404434+0000","last_active":"2024-09-21T02:07:31.404434+0000","last_peered":"2024-09-21T02:07:31.404434+0000","last_clean":"2024-09-21T02:07:31.404434+0000","last_became_active":"2024-09-21T02:03:58.069752+0000","last_became_peered":"2024-09-21T02:03:58.069752+0000","last_unstale":"2024-09-21T02:07:31.404434+0000","last_undegraded":"2024-09-21T02:07:31.404434+0000","last_fullsized":"2024-09-21T02:07:31.404434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'150","last_scrub_stamp":"2024-09-21T02:07:31.404285+0000","last_deep_scrub":"23'150","last_deep_scrub_stamp":"2024-09-21T02:07:31.404285+0000","last_clean_scrub_stamp":"2024-09-21T02:07:31.404285+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:29.420128+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'207","reported_seq":233,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:30.366526+0000","last_change":"2024-09-21T02:07:30.366526+0000","last_active":"2024-09-21T02:07:30.366526+0000","last_peered":"2024-09-21T02:07:30.366526+0000","last_clean":"2024-09-21T02:07:30.366526+0000","last_became_active":"2024-09-21T02:03:58.073057+0000","last_became_peered":"2024-09-21T02:03:58.073057+0000","last_unstale":"2024-09-21T02:07:30.366526+0000","last_undegraded":"2024-09-21T02:07:30.366526+0000","last_fullsized":"2024-09-21T02:07:30.366526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'207","last_scrub_stamp":"2024-09-21T02:07:30.366446+0000","last_deep_scrub":"23'207","last_deep_scrub_stamp":"2024-09-21T02:07:30.366446+0000","last_clean_scrub_stamp":"2024-09-21T02:07:30.366446+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:28.068465+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'160","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:30.444819+0000","last_change":"2024-09-21T02:07:30.444819+0000","last_active":"2024-09-21T02:07:30.444819+0000","last_peered":"2024-09-21T02:07:30.444819+0000","last_clean":"2024-09-21T02:07:30.444819+0000","last_became_active":"2024-09-21T02:03:58.082289+0000","last_became_peered":"2024-09-21T02:03:58.082289+0000","last_unstale":"2024-09-21T02:07:30.444819+0000","last_undegraded":"2024-09-21T02:07:30.444819+0000","last_fullsized":"2024-09-21T02:07:30.444819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-09-21T02:07:30.444728+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-09-21T02:07:30.444728+0000","last_clean_scrub_stamp":"2024-09-21T02:07:30.444728+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:57.927995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'217","reported_seq":243,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:28.247499+0000","last_change":"2024-09-21T02:07:28.247499+0000","last_active":"2024-09-21T02:07:28.247499+0000","last_peered":"2024-09-21T02:07:28.247499+0000","last_clean":"2024-09-21T02:07:28.247499+0000","last_became_active":"2024-09-21T02:03:58.083050+0000","last_became_peered":"2024-09-21T02:03:58.083050+0000","last_unstale":"2024-09-21T02:07:28.247499+0000","last_undegraded":"2024-09-21T02:07:28.247499+0000","last_fullsized":"2024-09-21T02:07:28.247499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'217","last_scrub_stamp":"2024-09-21T02:07:28.247369+0000","last_deep_scrub":"23'217","last_deep_scrub_stamp":"2024-09-21T02:07:28.247369+0000","last_clean_scrub_stamp":"2024-09-21T02:07:28.247369+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:37.229199+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'166","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:29.039563+0000","last_change":"2024-09-21T02:07:29.039563+0000","last_active":"2024-09-21T02:07:29.039563+0000","last_peered":"2024-09-21T02:07:29.039563+0000","last_clean":"2024-09-21T02:07:29.039563+0000","last_became_active":"2024-09-21T02:03:58.074152+0000","last_became_peered":"2024-09-21T02:03:58.074152+0000","last_unstale":"2024-09-21T02:07:29.039563+0000","last_undegraded":"2024-09-21T02:07:29.039563+0000","last_fullsized":"2024-09-21T02:07:29.039563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'166","last_scrub_stamp":"2024-09-21T02:07:29.039465+0000","last_deep_scrub":"23'166","last_deep_scrub_stamp":"2024-09-21T02:07:29.039465+0000","last_clean_scrub_stamp":"2024-09-21T02:07:29.039465+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:14.354277+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'152","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:29.479495+0000","last_change":"2024-09-21T02:07:29.479495+0000","last_active":"2024-09-21T02:07:29.479495+0000","last_peered":"2024-09-21T02:07:29.479495+0000","last_clean":"2024-09-21T02:07:29.479495+0000","last_became_active":"2024-09-21T02:03:58.081606+0000","last_became_peered":"2024-09-21T02:03:58.081606+0000","last_unstale":"2024-09-21T02:07:29.479495+0000","last_undegraded":"2024-09-21T02:07:29.479495+0000","last_fullsized":"2024-09-21T02:07:29.479495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'152","last_scrub_stamp":"2024-09-21T02:07:29.479400+0000","last_deep_scrub":"23'152","last_deep_scrub_stamp":"2024-09-21T02:07:29.479400+0000","last_clean_scrub_stamp":"2024-09-21T02:07:29.479400+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:12.876370+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'134","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:29.390415+0000","last_change":"2024-09-21T02:07:29.390415+0000","last_active":"2024-09-21T02:07:29.390415+0000","last_peered":"2024-09-21T02:07:29.390415+0000","last_clean":"2024-09-21T02:07:29.390415+0000","last_became_active":"2024-09-21T02:03:58.080840+0000","last_became_peered":"2024-09-21T02:03:58.080840+0000","last_unstale":"2024-09-21T02:07:29.390415+0000","last_undegraded":"2024-09-21T02:07:29.390415+0000","last_fullsized":"2024-09-21T02:07:29.390415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'134","last_scrub_stamp":"2024-09-21T02:07:29.390318+0000","last_deep_scrub":"23'134","last_deep_scrub_stamp":"2024-09-21T02:07:29.390318+0000","last_clean_scrub_stamp":"2024-09-21T02:07:29.390318+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:29.727094+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'153","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:31.349411+0000","last_change":"2024-09-21T02:07:31.349411+0000","last_active":"2024-09-21T02:07:31.349411+0000","last_peered":"2024-09-21T02:07:31.349411+0000","last_clean":"2024-09-21T02:07:31.349411+0000","last_became_active":"2024-09-21T02:03:58.078304+0000","last_became_peered":"2024-09-21T02:03:58.078304+0000","last_unstale":"2024-09-21T02:07:31.349411+0000","last_undegraded":"2024-09-21T02:07:31.349411+0000","last_fullsized":"2024-09-21T02:07:31.349411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'153","last_scrub_stamp":"2024-09-21T02:07:31.349307+0000","last_deep_scrub":"23'153","last_deep_scrub_stamp":"2024-09-21T02:07:31.349307+0000","last_clean_scrub_stamp":"2024-09-21T02:07:31.349307+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:51.337660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'158","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.357002+0000","last_change":"2024-09-21T02:07:25.357002+0000","last_active":"2024-09-21T02:07:25.357002+0000","last_peered":"2024-09-21T02:07:25.357002+0000","last_clean":"2024-09-21T02:07:25.357002+0000","last_became_active":"2024-09-21T02:03:58.077401+0000","last_became_peered":"2024-09-21T02:03:58.077401+0000","last_unstale":"2024-09-21T02:07:25.357002+0000","last_undegraded":"2024-09-21T02:07:25.357002+0000","last_fullsized":"2024-09-21T02:07:25.357002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'158","last_scrub_stamp":"2024-09-21T02:07:25.356899+0000","last_deep_scrub":"23'158","last_deep_scrub_stamp":"2024-09-21T02:07:25.356899+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.356899+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:33.430885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'180","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:30.290046+0000","last_change":"2024-09-21T02:07:30.290046+0000","last_active":"2024-09-21T02:07:30.290046+0000","last_peered":"2024-09-21T02:07:30.290046+0000","last_clean":"2024-09-21T02:07:30.290046+0000","last_became_active":"2024-09-21T02:03:58.084801+0000","last_became_peered":"2024-09-21T02:03:58.084801+0000","last_unstale":"2024-09-21T02:07:30.290046+0000","last_undegraded":"2024-09-21T02:07:30.290046+0000","last_fullsized":"2024-09-21T02:07:30.290046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'180","last_scrub_stamp":"2024-09-21T02:07:30.289953+0000","last_deep_scrub":"23'180","last_deep_scrub_stamp":"2024-09-21T02:07:30.289953+0000","last_clean_scrub_stamp":"2024-09-21T02:07:30.289953+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:57:52.615400+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'149","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.086572+0000","last_change":"2024-09-21T02:07:25.086572+0000","last_active":"2024-09-21T02:07:25.086572+0000","last_peered":"2024-09-21T02:07:25.086572+0000","last_clean":"2024-09-21T02:07:25.086572+0000","last_became_active":"2024-09-21T02:03:58.067209+0000","last_became_peered":"2024-09-21T02:03:58.067209+0000","last_unstale":"2024-09-21T02:07:25.086572+0000","last_undegraded":"2024-09-21T02:07:25.086572+0000","last_fullsized":"2024-09-21T02:07:25.086572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'149","last_scrub_stamp":"2024-09-21T02:07:25.086473+0000","last_deep_scrub":"23'149","last_deep_scrub_stamp":"2024-09-21T02:07:25.086473+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.086473+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:35.389545+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1647","reported_seq":1675,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:21.495080+0000","last_change":"2024-09-21T02:07:21.495080+0000","last_active":"2024-09-21T02:07:21.495080+0000","last_peered":"2024-09-21T02:07:21.495080+0000","last_clean":"2024-09-21T02:07:21.495080+0000","last_became_active":"2024-09-21T02:03:57.080518+0000","last_became_peered":"2024-09-21T02:03:57.080518+0000","last_unstale":"2024-09-21T02:07:21.495080+0000","last_undegraded":"2024-09-21T02:07:21.495080+0000","last_fullsized":"2024-09-21T02:07:21.495080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'1647","last_scrub_stamp":"2024-09-21T02:07:21.494984+0000","last_deep_scrub":"23'1647","last_deep_scrub_stamp":"2024-09-21T02:07:21.494984+0000","last_clean_scrub_stamp":"2024-09-21T02:07:21.494984+0000","objects_scrubbed":3,"log_size":1647,"log_dups_size":0,"ondisk_log_size":1647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:58.113368+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3425089,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:27.361744+0000","last_change":"2024-09-21T02:07:27.361744+0000","last_active":"2024-09-21T02:07:27.361744+0000","last_peered":"2024-09-21T02:07:27.361744+0000","last_clean":"2024-09-21T02:07:27.361744+0000","last_became_active":"2024-09-21T02:03:58.080022+0000","last_became_peered":"2024-09-21T02:03:58.080022+0000","last_unstale":"2024-09-21T02:07:27.361744+0000","last_undegraded":"2024-09-21T02:07:27.361744+0000","last_fullsized":"2024-09-21T02:07:27.361744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-09-21T02:07:27.361613+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-09-21T02:07:27.361613+0000","last_clean_scrub_stamp":"2024-09-21T02:07:27.361613+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:44.571220+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'4","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.231735+0000","last_change":"2024-09-21T02:07:26.231735+0000","last_active":"2024-09-21T02:07:26.231735+0000","last_peered":"2024-09-21T02:07:26.231735+0000","last_clean":"2024-09-21T02:07:26.231735+0000","last_became_active":"2024-09-21T02:03:57.076334+0000","last_became_peered":"2024-09-21T02:03:57.076334+0000","last_unstale":"2024-09-21T02:07:26.231735+0000","last_undegraded":"2024-09-21T02:07:26.231735+0000","last_fullsized":"2024-09-21T02:07:26.231735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-09-21T02:07:26.231607+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-09-21T02:07:26.231607+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.231607+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:55.006687+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'3","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:20.494661+0000","last_change":"2024-09-21T02:07:20.494661+0000","last_active":"2024-09-21T02:07:20.494661+0000","last_peered":"2024-09-21T02:07:20.494661+0000","last_clean":"2024-09-21T02:07:20.494661+0000","last_became_active":"2024-09-21T02:03:57.082899+0000","last_became_peered":"2024-09-21T02:03:57.082899+0000","last_unstale":"2024-09-21T02:07:20.494661+0000","last_undegraded":"2024-09-21T02:07:20.494661+0000","last_fullsized":"2024-09-21T02:07:20.494661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-09-21T02:07:20.494546+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-09-21T02:07:20.494546+0000","last_clean_scrub_stamp":"2024-09-21T02:07:20.494546+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:07.817601+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'190","reported_seq":216,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:27.500359+0000","last_change":"2024-09-21T02:07:27.500359+0000","last_active":"2024-09-21T02:07:27.500359+0000","last_peered":"2024-09-21T02:07:27.500359+0000","last_clean":"2024-09-21T02:07:27.500359+0000","last_became_active":"2024-09-21T02:03:58.082937+0000","last_became_peered":"2024-09-21T02:03:58.082937+0000","last_unstale":"2024-09-21T02:07:27.500359+0000","last_undegraded":"2024-09-21T02:07:27.500359+0000","last_fullsized":"2024-09-21T02:07:27.500359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'190","last_scrub_stamp":"2024-09-21T02:07:27.500237+0000","last_deep_scrub":"23'190","last_deep_scrub_stamp":"2024-09-21T02:07:27.500237+0000","last_clean_scrub_stamp":"2024-09-21T02:07:27.500237+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:47.621458+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.345230+0000","last_change":"2024-09-21T02:07:25.345230+0000","last_active":"2024-09-21T02:07:25.345230+0000","last_peered":"2024-09-21T02:07:25.345230+0000","last_clean":"2024-09-21T02:07:25.345230+0000","last_became_active":"2024-09-21T02:03:57.072054+0000","last_became_peered":"2024-09-21T02:03:57.072054+0000","last_unstale":"2024-09-21T02:07:25.345230+0000","last_undegraded":"2024-09-21T02:07:25.345230+0000","last_fullsized":"2024-09-21T02:07:25.345230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-21T02:07:25.345132+0000","last_deep_scrub":"21'1","last_deep_scrub_stamp":"2024-09-21T02:07:25.345132+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.345132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:42.018533+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'187","reported_seq":213,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:27.062765+0000","last_change":"2024-09-21T02:07:27.062765+0000","last_active":"2024-09-21T02:07:27.062765+0000","last_peered":"2024-09-21T02:07:27.062765+0000","last_clean":"2024-09-21T02:07:27.062765+0000","last_became_active":"2024-09-21T02:03:58.070960+0000","last_became_peered":"2024-09-21T02:03:58.070960+0000","last_unstale":"2024-09-21T02:07:27.062765+0000","last_undegraded":"2024-09-21T02:07:27.062765+0000","last_fullsized":"2024-09-21T02:07:27.062765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'187","last_scrub_stamp":"2024-09-21T02:07:27.062672+0000","last_deep_scrub":"23'187","last_deep_scrub_stamp":"2024-09-21T02:07:27.062672+0000","last_clean_scrub_stamp":"2024-09-21T02:07:27.062672+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:46.483877+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'6","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.277775+0000","last_change":"2024-09-21T02:07:25.277775+0000","last_active":"2024-09-21T02:07:25.277775+0000","last_peered":"2024-09-21T02:07:25.277775+0000","last_clean":"2024-09-21T02:07:25.277775+0000","last_became_active":"2024-09-21T02:03:57.078565+0000","last_became_peered":"2024-09-21T02:03:57.078565+0000","last_unstale":"2024-09-21T02:07:25.277775+0000","last_undegraded":"2024-09-21T02:07:25.277775+0000","last_fullsized":"2024-09-21T02:07:25.277775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-09-21T02:07:25.277666+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-09-21T02:07:25.277666+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.277666+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:38.722182+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'251","reported_seq":271,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.350335+0000","last_change":"2024-09-21T02:07:24.350335+0000","last_active":"2024-09-21T02:07:24.350335+0000","last_peered":"2024-09-21T02:07:24.350335+0000","last_clean":"2024-09-21T02:07:24.350335+0000","last_became_active":"2024-09-21T02:04:45.211981+0000","last_became_peered":"2024-09-21T02:04:45.211981+0000","last_unstale":"2024-09-21T02:07:24.350335+0000","last_undegraded":"2024-09-21T02:07:24.350335+0000","last_fullsized":"2024-09-21T02:07:24.350335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'251","last_scrub_stamp":"2024-09-21T02:07:24.350186+0000","last_deep_scrub":"23'251","last_deep_scrub_stamp":"2024-09-21T02:07:24.350186+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.350186+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:37.304580+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.390365+0000","last_change":"2024-09-21T02:07:24.390365+0000","last_active":"2024-09-21T02:07:24.390365+0000","last_peered":"2024-09-21T02:07:24.390365+0000","last_clean":"2024-09-21T02:07:24.390365+0000","last_became_active":"2024-09-21T02:03:57.074548+0000","last_became_peered":"2024-09-21T02:03:57.074548+0000","last_unstale":"2024-09-21T02:07:24.390365+0000","last_undegraded":"2024-09-21T02:07:24.390365+0000","last_fullsized":"2024-09-21T02:07:24.390365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-09-21T02:07:24.390261+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-09-21T02:07:24.390261+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.390261+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:02:16.799530+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'160","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.331248+0000","last_change":"2024-09-21T02:07:23.331248+0000","last_active":"2024-09-21T02:07:23.331248+0000","last_peered":"2024-09-21T02:07:23.331248+0000","last_clean":"2024-09-21T02:07:23.331248+0000","last_became_active":"2024-09-21T02:03:58.079911+0000","last_became_peered":"2024-09-21T02:03:58.079911+0000","last_unstale":"2024-09-21T02:07:23.331248+0000","last_undegraded":"2024-09-21T02:07:23.331248+0000","last_fullsized":"2024-09-21T02:07:23.331248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-09-21T02:07:23.331148+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-09-21T02:07:23.331148+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.331148+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:27:45.596883+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'12","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.038926+0000","last_change":"2024-09-21T02:07:25.038926+0000","last_active":"2024-09-21T02:07:25.038926+0000","last_peered":"2024-09-21T02:07:25.038926+0000","last_clean":"2024-09-21T02:07:25.038926+0000","last_became_active":"2024-09-21T02:03:57.077530+0000","last_became_peered":"2024-09-21T02:03:57.077530+0000","last_unstale":"2024-09-21T02:07:25.038926+0000","last_undegraded":"2024-09-21T02:07:25.038926+0000","last_fullsized":"2024-09-21T02:07:25.038926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-09-21T02:07:25.038824+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-09-21T02:07:25.038824+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.038824+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.443956+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'168","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.077677+0000","last_change":"2024-09-21T02:07:24.077677+0000","last_active":"2024-09-21T02:07:24.077677+0000","last_peered":"2024-09-21T02:07:24.077677+0000","last_clean":"2024-09-21T02:07:24.077677+0000","last_became_active":"2024-09-21T02:03:58.074005+0000","last_became_peered":"2024-09-21T02:03:58.074005+0000","last_unstale":"2024-09-21T02:07:24.077677+0000","last_undegraded":"2024-09-21T02:07:24.077677+0000","last_fullsized":"2024-09-21T02:07:24.077677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'168","last_scrub_stamp":"2024-09-21T02:07:24.077579+0000","last_deep_scrub":"23'168","last_deep_scrub_stamp":"2024-09-21T02:07:24.077579+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.077579+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:56.654943+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'2022","reported_seq":2050,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.334809+0000","last_change":"2024-09-21T02:07:24.334809+0000","last_active":"2024-09-21T02:07:24.334809+0000","last_peered":"2024-09-21T02:07:24.334809+0000","last_clean":"2024-09-21T02:07:24.334809+0000","last_became_active":"2024-09-21T02:03:57.077413+0000","last_became_peered":"2024-09-21T02:03:57.077413+0000","last_unstale":"2024-09-21T02:07:24.334809+0000","last_undegraded":"2024-09-21T02:07:24.334809+0000","last_fullsized":"2024-09-21T02:07:24.334809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2022","last_scrub_stamp":"2024-09-21T02:07:24.334705+0000","last_deep_scrub":"23'2022","last_deep_scrub_stamp":"2024-09-21T02:07:24.334705+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.334705+0000","objects_scrubbed":5,"log_size":2022,"log_dups_size":0,"ondisk_log_size":2022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:00.215663+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2018,"num_write_kb":6033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'176","reported_seq":208,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:33.715365+0000","last_change":"2024-09-21T02:07:33.715365+0000","last_active":"2024-09-21T02:07:33.715365+0000","last_peered":"2024-09-21T02:07:33.715365+0000","last_clean":"2024-09-21T02:07:33.715365+0000","last_became_active":"2024-09-21T02:04:45.213182+0000","last_became_peered":"2024-09-21T02:04:45.213182+0000","last_unstale":"2024-09-21T02:07:33.715365+0000","last_undegraded":"2024-09-21T02:07:33.715365+0000","last_fullsized":"2024-09-21T02:07:33.715365+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'176","last_scrub_stamp":"2024-09-21T02:07:33.715243+0000","last_deep_scrub":"23'176","last_deep_scrub_stamp":"2024-09-21T02:07:33.715243+0000","last_clean_scrub_stamp":"2024-09-21T02:07:33.715243+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:03.265264+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.347581+0000","last_change":"2024-09-21T02:07:26.347581+0000","last_active":"2024-09-21T02:07:26.347581+0000","last_peered":"2024-09-21T02:07:26.347581+0000","last_clean":"2024-09-21T02:07:26.347581+0000","last_became_active":"2024-09-21T02:03:57.071186+0000","last_became_peered":"2024-09-21T02:03:57.071186+0000","last_unstale":"2024-09-21T02:07:26.347581+0000","last_undegraded":"2024-09-21T02:07:26.347581+0000","last_fullsized":"2024-09-21T02:07:26.347581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-09-21T02:07:26.347501+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-09-21T02:07:26.347501+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.347501+0000","objects_scrubbed":4,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:11.899935+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'177","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.362931+0000","last_change":"2024-09-21T02:07:22.362931+0000","last_active":"2024-09-21T02:07:22.362931+0000","last_peered":"2024-09-21T02:07:22.362931+0000","last_clean":"2024-09-21T02:07:22.362931+0000","last_became_active":"2024-09-21T02:03:58.078742+0000","last_became_peered":"2024-09-21T02:03:58.078742+0000","last_unstale":"2024-09-21T02:07:22.362931+0000","last_undegraded":"2024-09-21T02:07:22.362931+0000","last_fullsized":"2024-09-21T02:07:22.362931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-09-21T02:07:22.362848+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-09-21T02:07:22.362848+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.362848+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:27.077504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'4","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.380539+0000","last_change":"2024-09-21T02:07:23.380539+0000","last_active":"2024-09-21T02:07:23.380539+0000","last_peered":"2024-09-21T02:07:23.380539+0000","last_clean":"2024-09-21T02:07:23.380539+0000","last_became_active":"2024-09-21T02:03:57.073754+0000","last_became_peered":"2024-09-21T02:03:57.073754+0000","last_unstale":"2024-09-21T02:07:23.380539+0000","last_undegraded":"2024-09-21T02:07:23.380539+0000","last_fullsized":"2024-09-21T02:07:23.380539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-09-21T02:07:23.380430+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-09-21T02:07:23.380430+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.380430+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:20:15.177793+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'201","reported_seq":227,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:34.744363+0000","last_change":"2024-09-21T02:07:34.744363+0000","last_active":"2024-09-21T02:07:34.744363+0000","last_peered":"2024-09-21T02:07:34.744363+0000","last_clean":"2024-09-21T02:07:34.744363+0000","last_became_active":"2024-09-21T02:03:58.087476+0000","last_became_peered":"2024-09-21T02:03:58.087476+0000","last_unstale":"2024-09-21T02:07:34.744363+0000","last_undegraded":"2024-09-21T02:07:34.744363+0000","last_fullsized":"2024-09-21T02:07:34.744363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'201","last_scrub_stamp":"2024-09-21T02:07:34.744262+0000","last_deep_scrub":"23'201","last_deep_scrub_stamp":"2024-09-21T02:07:34.744262+0000","last_clean_scrub_stamp":"2024-09-21T02:07:34.744262+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:30.801286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'6","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.362036+0000","last_change":"2024-09-21T02:07:23.362036+0000","last_active":"2024-09-21T02:07:23.362036+0000","last_peered":"2024-09-21T02:07:23.362036+0000","last_clean":"2024-09-21T02:07:23.362036+0000","last_became_active":"2024-09-21T02:03:57.076758+0000","last_became_peered":"2024-09-21T02:03:57.076758+0000","last_unstale":"2024-09-21T02:07:23.362036+0000","last_undegraded":"2024-09-21T02:07:23.362036+0000","last_fullsized":"2024-09-21T02:07:23.362036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-09-21T02:07:23.361925+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-09-21T02:07:23.361925+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.361925+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:06.471965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":985,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'197","reported_seq":223,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:29.279563+0000","last_change":"2024-09-21T02:07:29.279563+0000","last_active":"2024-09-21T02:07:29.279563+0000","last_peered":"2024-09-21T02:07:29.279563+0000","last_clean":"2024-09-21T02:07:29.279563+0000","last_became_active":"2024-09-21T02:03:58.083773+0000","last_became_peered":"2024-09-21T02:03:58.083773+0000","last_unstale":"2024-09-21T02:07:29.279563+0000","last_undegraded":"2024-09-21T02:07:29.279563+0000","last_fullsized":"2024-09-21T02:07:29.279563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'197","last_scrub_stamp":"2024-09-21T02:07:29.279454+0000","last_deep_scrub":"23'197","last_deep_scrub_stamp":"2024-09-21T02:07:29.279454+0000","last_clean_scrub_stamp":"2024-09-21T02:07:29.279454+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:06.343061+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'24","reported_seq":52,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.341665+0000","last_change":"2024-09-21T02:07:23.341665+0000","last_active":"2024-09-21T02:07:23.341665+0000","last_peered":"2024-09-21T02:07:23.341665+0000","last_clean":"2024-09-21T02:07:23.341665+0000","last_became_active":"2024-09-21T02:03:57.078377+0000","last_became_peered":"2024-09-21T02:03:57.078377+0000","last_unstale":"2024-09-21T02:07:23.341665+0000","last_undegraded":"2024-09-21T02:07:23.341665+0000","last_fullsized":"2024-09-21T02:07:23.341665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'24","last_scrub_stamp":"2024-09-21T02:07:23.341571+0000","last_deep_scrub":"23'24","last_deep_scrub_stamp":"2024-09-21T02:07:23.341571+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.341571+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:40.407580+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'215","reported_seq":241,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:32.723020+0000","last_change":"2024-09-21T02:07:32.723020+0000","last_active":"2024-09-21T02:07:32.723020+0000","last_peered":"2024-09-21T02:07:32.723020+0000","last_clean":"2024-09-21T02:07:32.723020+0000","last_became_active":"2024-09-21T02:03:58.083515+0000","last_became_peered":"2024-09-21T02:03:58.083515+0000","last_unstale":"2024-09-21T02:07:32.723020+0000","last_undegraded":"2024-09-21T02:07:32.723020+0000","last_fullsized":"2024-09-21T02:07:32.723020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'215","last_scrub_stamp":"2024-09-21T02:07:32.722922+0000","last_deep_scrub":"23'215","last_deep_scrub_stamp":"2024-09-21T02:07:32.722922+0000","last_clean_scrub_stamp":"2024-09-21T02:07:32.722922+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:26.798929+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'5","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:21.102611+0000","last_change":"2024-09-21T02:07:21.102611+0000","last_active":"2024-09-21T02:07:21.102611+0000","last_peered":"2024-09-21T02:07:21.102611+0000","last_clean":"2024-09-21T02:07:21.102611+0000","last_became_active":"2024-09-21T02:03:57.066697+0000","last_became_peered":"2024-09-21T02:03:57.066697+0000","last_unstale":"2024-09-21T02:07:21.102611+0000","last_undegraded":"2024-09-21T02:07:21.102611+0000","last_fullsized":"2024-09-21T02:07:21.102611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-09-21T02:07:21.102504+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-09-21T02:07:21.102504+0000","last_clean_scrub_stamp":"2024-09-21T02:07:21.102504+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:27.790217+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'175","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:31.732723+0000","last_change":"2024-09-21T02:07:31.732723+0000","last_active":"2024-09-21T02:07:31.732723+0000","last_peered":"2024-09-21T02:07:31.732723+0000","last_clean":"2024-09-21T02:07:31.732723+0000","last_became_active":"2024-09-21T02:03:58.078764+0000","last_became_peered":"2024-09-21T02:03:58.078764+0000","last_unstale":"2024-09-21T02:07:31.732723+0000","last_undegraded":"2024-09-21T02:07:31.732723+0000","last_fullsized":"2024-09-21T02:07:31.732723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'175","last_scrub_stamp":"2024-09-21T02:07:31.732617+0000","last_deep_scrub":"23'175","last_deep_scrub_stamp":"2024-09-21T02:07:31.732617+0000","last_clean_scrub_stamp":"2024-09-21T02:07:31.732617+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:35.995322+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'3","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:20.262148+0000","last_change":"2024-09-21T02:07:20.262148+0000","last_active":"2024-09-21T02:07:20.262148+0000","last_peered":"2024-09-21T02:07:20.262148+0000","last_clean":"2024-09-21T02:07:20.262148+0000","last_became_active":"2024-09-21T02:03:57.071901+0000","last_became_peered":"2024-09-21T02:03:57.071901+0000","last_unstale":"2024-09-21T02:07:20.262148+0000","last_undegraded":"2024-09-21T02:07:20.262148+0000","last_fullsized":"2024-09-21T02:07:20.262148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-09-21T02:07:20.262038+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-09-21T02:07:20.262038+0000","last_clean_scrub_stamp":"2024-09-21T02:07:20.262038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:02.657858+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:30.751940+0000","last_change":"2024-09-21T02:07:30.751940+0000","last_active":"2024-09-21T02:07:30.751940+0000","last_peered":"2024-09-21T02:07:30.751940+0000","last_clean":"2024-09-21T02:07:30.751940+0000","last_became_active":"2024-09-21T02:03:58.079586+0000","last_became_peered":"2024-09-21T02:03:58.079586+0000","last_unstale":"2024-09-21T02:07:30.751940+0000","last_undegraded":"2024-09-21T02:07:30.751940+0000","last_fullsized":"2024-09-21T02:07:30.751940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-09-21T02:07:30.751857+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-09-21T02:07:30.751857+0000","last_clean_scrub_stamp":"2024-09-21T02:07:30.751857+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:38.842251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'2","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.026162+0000","last_change":"2024-09-21T02:07:23.026162+0000","last_active":"2024-09-21T02:07:23.026162+0000","last_peered":"2024-09-21T02:07:23.026162+0000","last_clean":"2024-09-21T02:07:23.026162+0000","last_became_active":"2024-09-21T02:03:57.076075+0000","last_became_peered":"2024-09-21T02:03:57.076075+0000","last_unstale":"2024-09-21T02:07:23.026162+0000","last_undegraded":"2024-09-21T02:07:23.026162+0000","last_fullsized":"2024-09-21T02:07:23.026162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-21T02:07:23.026072+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-09-21T02:07:23.026072+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.026072+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:47.602084+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'165","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.354018+0000","last_change":"2024-09-21T02:07:26.354018+0000","last_active":"2024-09-21T02:07:26.354018+0000","last_peered":"2024-09-21T02:07:26.354018+0000","last_clean":"2024-09-21T02:07:26.354018+0000","last_became_active":"2024-09-21T02:03:58.079909+0000","last_became_peered":"2024-09-21T02:03:58.079909+0000","last_unstale":"2024-09-21T02:07:26.354018+0000","last_undegraded":"2024-09-21T02:07:26.354018+0000","last_fullsized":"2024-09-21T02:07:26.354018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-09-21T02:07:26.353897+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-09-21T02:07:26.353897+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.353897+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:24.495042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:21.278053+0000","last_change":"2024-09-21T02:07:21.278053+0000","last_active":"2024-09-21T02:07:21.278053+0000","last_peered":"2024-09-21T02:07:21.278053+0000","last_clean":"2024-09-21T02:07:21.278053+0000","last_became_active":"2024-09-21T02:03:57.074395+0000","last_became_peered":"2024-09-21T02:03:57.074395+0000","last_unstale":"2024-09-21T02:07:21.278053+0000","last_undegraded":"2024-09-21T02:07:21.278053+0000","last_fullsized":"2024-09-21T02:07:21.278053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-09-21T02:07:21.277976+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-09-21T02:07:21.277976+0000","last_clean_scrub_stamp":"2024-09-21T02:07:21.277976+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:15.449004+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'175","reported_seq":201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:29.731149+0000","last_change":"2024-09-21T02:07:29.731149+0000","last_active":"2024-09-21T02:07:29.731149+0000","last_peered":"2024-09-21T02:07:29.731149+0000","last_clean":"2024-09-21T02:07:29.731149+0000","last_became_active":"2024-09-21T02:03:58.087640+0000","last_became_peered":"2024-09-21T02:03:58.087640+0000","last_unstale":"2024-09-21T02:07:29.731149+0000","last_undegraded":"2024-09-21T02:07:29.731149+0000","last_fullsized":"2024-09-21T02:07:29.731149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'175","last_scrub_stamp":"2024-09-21T02:07:29.731028+0000","last_deep_scrub":"23'175","last_deep_scrub_stamp":"2024-09-21T02:07:29.731028+0000","last_clean_scrub_stamp":"2024-09-21T02:07:29.731028+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:30.990192+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'74","reported_seq":126,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.826788+0000","last_change":"2024-09-21T02:07:24.826788+0000","last_active":"2024-09-21T02:07:24.826788+0000","last_peered":"2024-09-21T02:07:24.826788+0000","last_clean":"2024-09-21T02:07:24.826788+0000","last_became_active":"2024-09-21T02:03:47.427088+0000","last_became_peered":"2024-09-21T02:03:47.427088+0000","last_unstale":"2024-09-21T02:07:24.826788+0000","last_undegraded":"2024-09-21T02:07:24.826788+0000","last_fullsized":"2024-09-21T02:07:24.826788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'74","last_scrub_stamp":"2024-09-21T02:07:24.826683+0000","last_deep_scrub":"23'74","last_deep_scrub_stamp":"2024-09-21T02:07:24.826683+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.826683+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:17.240670+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'6","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.430645+0000","last_change":"2024-09-21T02:07:22.430645+0000","last_active":"2024-09-21T02:07:22.430645+0000","last_peered":"2024-09-21T02:07:22.430645+0000","last_clean":"2024-09-21T02:07:22.430645+0000","last_became_active":"2024-09-21T02:03:57.071652+0000","last_became_peered":"2024-09-21T02:03:57.071652+0000","last_unstale":"2024-09-21T02:07:22.430645+0000","last_undegraded":"2024-09-21T02:07:22.430645+0000","last_fullsized":"2024-09-21T02:07:22.430645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-09-21T02:07:22.430517+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-09-21T02:07:22.430517+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.430517+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:59:30.768891+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'183","reported_seq":209,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.496557+0000","last_change":"2024-09-21T02:07:23.496557+0000","last_active":"2024-09-21T02:07:23.496557+0000","last_peered":"2024-09-21T02:07:23.496557+0000","last_clean":"2024-09-21T02:07:23.496557+0000","last_became_active":"2024-09-21T02:03:58.080860+0000","last_became_peered":"2024-09-21T02:03:58.080860+0000","last_unstale":"2024-09-21T02:07:23.496557+0000","last_undegraded":"2024-09-21T02:07:23.496557+0000","last_fullsized":"2024-09-21T02:07:23.496557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'183","last_scrub_stamp":"2024-09-21T02:07:23.496471+0000","last_deep_scrub":"23'183","last_deep_scrub_stamp":"2024-09-21T02:07:23.496471+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.496471+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:48.176438+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.836340+0000","last_change":"2024-09-21T02:07:26.836340+0000","last_active":"2024-09-21T02:07:26.836340+0000","last_peered":"2024-09-21T02:07:26.836340+0000","last_clean":"2024-09-21T02:07:26.836340+0000","last_became_active":"2024-09-21T02:03:57.080449+0000","last_became_peered":"2024-09-21T02:03:57.080449+0000","last_unstale":"2024-09-21T02:07:26.836340+0000","last_undegraded":"2024-09-21T02:07:26.836340+0000","last_fullsized":"2024-09-21T02:07:26.836340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-09-21T02:07:26.836236+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-09-21T02:07:26.836236+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.836236+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:15.343447+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'165","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:20.332299+0000","last_change":"2024-09-21T02:07:20.332299+0000","last_active":"2024-09-21T02:07:20.332299+0000","last_peered":"2024-09-21T02:07:20.332299+0000","last_clean":"2024-09-21T02:07:20.332299+0000","last_became_active":"2024-09-21T02:03:58.080356+0000","last_became_peered":"2024-09-21T02:03:58.080356+0000","last_unstale":"2024-09-21T02:07:20.332299+0000","last_undegraded":"2024-09-21T02:07:20.332299+0000","last_fullsized":"2024-09-21T02:07:20.332299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-09-21T02:07:20.332215+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-09-21T02:07:20.332215+0000","last_clean_scrub_stamp":"2024-09-21T02:07:20.332215+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:53.857052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'13","reported_seq":46,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.302636+0000","last_change":"2024-09-21T02:07:22.302636+0000","last_active":"2024-09-21T02:07:22.302636+0000","last_peered":"2024-09-21T02:07:22.302636+0000","last_clean":"2024-09-21T02:07:22.302636+0000","last_became_active":"2024-09-21T02:03:57.074918+0000","last_became_peered":"2024-09-21T02:03:57.074918+0000","last_unstale":"2024-09-21T02:07:22.302636+0000","last_undegraded":"2024-09-21T02:07:22.302636+0000","last_fullsized":"2024-09-21T02:07:22.302636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13","last_scrub_stamp":"2024-09-21T02:07:22.302536+0000","last_deep_scrub":"23'13","last_deep_scrub_stamp":"2024-09-21T02:07:22.302536+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.302536+0000","objects_scrubbed":3,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:50.711231+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":595,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'165","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.071336+0000","last_change":"2024-09-21T02:07:26.071336+0000","last_active":"2024-09-21T02:07:26.071336+0000","last_peered":"2024-09-21T02:07:26.071336+0000","last_clean":"2024-09-21T02:07:26.071336+0000","last_became_active":"2024-09-21T02:03:58.077131+0000","last_became_peered":"2024-09-21T02:03:58.077131+0000","last_unstale":"2024-09-21T02:07:26.071336+0000","last_undegraded":"2024-09-21T02:07:26.071336+0000","last_fullsized":"2024-09-21T02:07:26.071336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-09-21T02:07:26.071237+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-09-21T02:07:26.071237+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.071237+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:10.861862+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:21.410346+0000","last_change":"2024-09-21T02:07:21.410346+0000","last_active":"2024-09-21T02:07:21.410346+0000","last_peered":"2024-09-21T02:07:21.410346+0000","last_clean":"2024-09-21T02:07:21.410346+0000","last_became_active":"2024-09-21T02:03:57.077705+0000","last_became_peered":"2024-09-21T02:03:57.077705+0000","last_unstale":"2024-09-21T02:07:21.410346+0000","last_undegraded":"2024-09-21T02:07:21.410346+0000","last_fullsized":"2024-09-21T02:07:21.410346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-21T02:07:21.410235+0000","last_deep_scrub":"21'1","last_deep_scrub_stamp":"2024-09-21T02:07:21.410235+0000","last_clean_scrub_stamp":"2024-09-21T02:07:21.410235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:12:08.239699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'177","reported_seq":203,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.061460+0000","last_change":"2024-09-21T02:07:23.061460+0000","last_active":"2024-09-21T02:07:23.061460+0000","last_peered":"2024-09-21T02:07:23.061460+0000","last_clean":"2024-09-21T02:07:23.061460+0000","last_became_active":"2024-09-21T02:03:58.082554+0000","last_became_peered":"2024-09-21T02:03:58.082554+0000","last_unstale":"2024-09-21T02:07:23.061460+0000","last_undegraded":"2024-09-21T02:07:23.061460+0000","last_fullsized":"2024-09-21T02:07:23.061460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'177","last_scrub_stamp":"2024-09-21T02:07:23.061320+0000","last_deep_scrub":"23'177","last_deep_scrub_stamp":"2024-09-21T02:07:23.061320+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.061320+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:52.226487+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:20.043502+0000","last_change":"2024-09-21T02:07:20.043502+0000","last_active":"2024-09-21T02:07:20.043502+0000","last_peered":"2024-09-21T02:07:20.043502+0000","last_clean":"2024-09-21T02:07:20.043502+0000","last_became_active":"2024-09-21T02:03:57.080771+0000","last_became_peered":"2024-09-21T02:03:57.080771+0000","last_unstale":"2024-09-21T02:07:20.043502+0000","last_undegraded":"2024-09-21T02:07:20.043502+0000","last_fullsized":"2024-09-21T02:07:20.043502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:07:20.043406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:07:20.043406+0000","last_clean_scrub_stamp":"2024-09-21T02:07:20.043406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:58.631295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'160","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:27.354274+0000","last_change":"2024-09-21T02:07:27.354274+0000","last_active":"2024-09-21T02:07:27.354274+0000","last_peered":"2024-09-21T02:07:27.354274+0000","last_clean":"2024-09-21T02:07:27.354274+0000","last_became_active":"2024-09-21T02:03:58.078283+0000","last_became_peered":"2024-09-21T02:03:58.078283+0000","last_unstale":"2024-09-21T02:07:27.354274+0000","last_undegraded":"2024-09-21T02:07:27.354274+0000","last_fullsized":"2024-09-21T02:07:27.354274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-09-21T02:07:27.354175+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-09-21T02:07:27.354175+0000","last_clean_scrub_stamp":"2024-09-21T02:07:27.354175+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:35.503948+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:23.852084+0000","last_change":"2024-09-21T02:07:23.852084+0000","last_active":"2024-09-21T02:07:23.852084+0000","last_peered":"2024-09-21T02:07:23.852084+0000","last_clean":"2024-09-21T02:07:23.852084+0000","last_became_active":"2024-09-21T02:03:57.080964+0000","last_became_peered":"2024-09-21T02:03:57.080964+0000","last_unstale":"2024-09-21T02:07:23.852084+0000","last_undegraded":"2024-09-21T02:07:23.852084+0000","last_fullsized":"2024-09-21T02:07:23.852084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-21T02:07:23.851959+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-09-21T02:07:23.851959+0000","last_clean_scrub_stamp":"2024-09-21T02:07:23.851959+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:18.402426+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:28.060343+0000","last_change":"2024-09-21T02:07:28.060343+0000","last_active":"2024-09-21T02:07:28.060343+0000","last_peered":"2024-09-21T02:07:28.060343+0000","last_clean":"2024-09-21T02:07:28.060343+0000","last_became_active":"2024-09-21T02:03:58.077171+0000","last_became_peered":"2024-09-21T02:03:58.077171+0000","last_unstale":"2024-09-21T02:07:28.060343+0000","last_undegraded":"2024-09-21T02:07:28.060343+0000","last_fullsized":"2024-09-21T02:07:28.060343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-09-21T02:07:28.060241+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-09-21T02:07:28.060241+0000","last_clean_scrub_stamp":"2024-09-21T02:07:28.060241+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:42.327510+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'23","reported_seq":51,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:21.057242+0000","last_change":"2024-09-21T02:07:21.057242+0000","last_active":"2024-09-21T02:07:21.057242+0000","last_peered":"2024-09-21T02:07:21.057242+0000","last_clean":"2024-09-21T02:07:21.057242+0000","last_became_active":"2024-09-21T02:03:57.077672+0000","last_became_peered":"2024-09-21T02:03:57.077672+0000","last_unstale":"2024-09-21T02:07:21.057242+0000","last_undegraded":"2024-09-21T02:07:21.057242+0000","last_fullsized":"2024-09-21T02:07:21.057242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'23","last_scrub_stamp":"2024-09-21T02:07:21.057165+0000","last_deep_scrub":"23'23","last_deep_scrub_stamp":"2024-09-21T02:07:21.057165+0000","last_clean_scrub_stamp":"2024-09-21T02:07:21.057165+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:26.553979+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'166","reported_seq":192,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.453511+0000","last_change":"2024-09-21T02:07:24.453511+0000","last_active":"2024-09-21T02:07:24.453511+0000","last_peered":"2024-09-21T02:07:24.453511+0000","last_clean":"2024-09-21T02:07:24.453511+0000","last_became_active":"2024-09-21T02:03:58.074607+0000","last_became_peered":"2024-09-21T02:03:58.074607+0000","last_unstale":"2024-09-21T02:07:24.453511+0000","last_undegraded":"2024-09-21T02:07:24.453511+0000","last_fullsized":"2024-09-21T02:07:24.453511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'166","last_scrub_stamp":"2024-09-21T02:07:24.453418+0000","last_deep_scrub":"23'166","last_deep_scrub_stamp":"2024-09-21T02:07:24.453418+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.453418+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:09.468021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'7","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.404301+0000","last_change":"2024-09-21T02:07:22.404301+0000","last_active":"2024-09-21T02:07:22.404301+0000","last_peered":"2024-09-21T02:07:22.404301+0000","last_clean":"2024-09-21T02:07:22.404301+0000","last_became_active":"2024-09-21T02:03:57.079738+0000","last_became_peered":"2024-09-21T02:03:57.079738+0000","last_unstale":"2024-09-21T02:07:22.404301+0000","last_undegraded":"2024-09-21T02:07:22.404301+0000","last_fullsized":"2024-09-21T02:07:22.404301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-09-21T02:07:22.404202+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-09-21T02:07:22.404202+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.404202+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:27:06.969809+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'183","reported_seq":209,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:21.312998+0000","last_change":"2024-09-21T02:07:21.312998+0000","last_active":"2024-09-21T02:07:21.312998+0000","last_peered":"2024-09-21T02:07:21.312998+0000","last_clean":"2024-09-21T02:07:21.312998+0000","last_became_active":"2024-09-21T02:03:58.077668+0000","last_became_peered":"2024-09-21T02:03:58.077668+0000","last_unstale":"2024-09-21T02:07:21.312998+0000","last_undegraded":"2024-09-21T02:07:21.312998+0000","last_fullsized":"2024-09-21T02:07:21.312998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'183","last_scrub_stamp":"2024-09-21T02:07:21.312921+0000","last_deep_scrub":"23'183","last_deep_scrub_stamp":"2024-09-21T02:07:21.312921+0000","last_clean_scrub_stamp":"2024-09-21T02:07:21.312921+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:45.736859+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'3","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:28.770411+0000","last_change":"2024-09-21T02:07:28.770411+0000","last_active":"2024-09-21T02:07:28.770411+0000","last_peered":"2024-09-21T02:07:28.770411+0000","last_clean":"2024-09-21T02:07:28.770411+0000","last_became_active":"2024-09-21T02:03:57.081903+0000","last_became_peered":"2024-09-21T02:03:57.081903+0000","last_unstale":"2024-09-21T02:07:28.770411+0000","last_undegraded":"2024-09-21T02:07:28.770411+0000","last_fullsized":"2024-09-21T02:07:28.770411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-09-21T02:07:28.770317+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-09-21T02:07:28.770317+0000","last_clean_scrub_stamp":"2024-09-21T02:07:28.770317+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:11.434573+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.078114+0000","last_change":"2024-09-21T02:07:22.078114+0000","last_active":"2024-09-21T02:07:22.078114+0000","last_peered":"2024-09-21T02:07:22.078114+0000","last_clean":"2024-09-21T02:07:22.078114+0000","last_became_active":"2024-09-21T02:03:58.081112+0000","last_became_peered":"2024-09-21T02:03:58.081112+0000","last_unstale":"2024-09-21T02:07:22.078114+0000","last_undegraded":"2024-09-21T02:07:22.078114+0000","last_fullsized":"2024-09-21T02:07:22.078114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-09-21T02:07:22.078019+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-09-21T02:07:22.078019+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.078019+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:54.231800+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'5","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.039499+0000","last_change":"2024-09-21T02:07:22.039499+0000","last_active":"2024-09-21T02:07:22.039499+0000","last_peered":"2024-09-21T02:07:22.039499+0000","last_clean":"2024-09-21T02:07:22.039499+0000","last_became_active":"2024-09-21T02:03:57.078251+0000","last_became_peered":"2024-09-21T02:03:57.078251+0000","last_unstale":"2024-09-21T02:07:22.039499+0000","last_undegraded":"2024-09-21T02:07:22.039499+0000","last_fullsized":"2024-09-21T02:07:22.039499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-09-21T02:07:22.039418+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-09-21T02:07:22.039418+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.039418+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:39.955029+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'173","reported_seq":199,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.495641+0000","last_change":"2024-09-21T02:07:25.495641+0000","last_active":"2024-09-21T02:07:25.495641+0000","last_peered":"2024-09-21T02:07:25.495641+0000","last_clean":"2024-09-21T02:07:25.495641+0000","last_became_active":"2024-09-21T02:03:58.080079+0000","last_became_peered":"2024-09-21T02:03:58.080079+0000","last_unstale":"2024-09-21T02:07:25.495641+0000","last_undegraded":"2024-09-21T02:07:25.495641+0000","last_fullsized":"2024-09-21T02:07:25.495641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'173","last_scrub_stamp":"2024-09-21T02:07:25.495515+0000","last_deep_scrub":"23'173","last_deep_scrub_stamp":"2024-09-21T02:07:25.495515+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.495515+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:39.302185+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'2","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.028729+0000","last_change":"2024-09-21T02:07:24.028729+0000","last_active":"2024-09-21T02:07:24.028729+0000","last_peered":"2024-09-21T02:07:24.028729+0000","last_clean":"2024-09-21T02:07:24.028729+0000","last_became_active":"2024-09-21T02:03:57.079074+0000","last_became_peered":"2024-09-21T02:03:57.079074+0000","last_unstale":"2024-09-21T02:07:24.028729+0000","last_undegraded":"2024-09-21T02:07:24.028729+0000","last_fullsized":"2024-09-21T02:07:24.028729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-21T02:07:24.028627+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-09-21T02:07:24.028627+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.028627+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:12.656747+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'164","reported_seq":190,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:26.497426+0000","last_change":"2024-09-21T02:07:26.497426+0000","last_active":"2024-09-21T02:07:26.497426+0000","last_peered":"2024-09-21T02:07:26.497426+0000","last_clean":"2024-09-21T02:07:26.497426+0000","last_became_active":"2024-09-21T02:03:58.083024+0000","last_became_peered":"2024-09-21T02:03:58.083024+0000","last_unstale":"2024-09-21T02:07:26.497426+0000","last_undegraded":"2024-09-21T02:07:26.497426+0000","last_fullsized":"2024-09-21T02:07:26.497426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'164","last_scrub_stamp":"2024-09-21T02:07:26.497351+0000","last_deep_scrub":"23'164","last_deep_scrub_stamp":"2024-09-21T02:07:26.497351+0000","last_clean_scrub_stamp":"2024-09-21T02:07:26.497351+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:17.101219+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'7","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:27.805484+0000","last_change":"2024-09-21T02:07:27.805484+0000","last_active":"2024-09-21T02:07:27.805484+0000","last_peered":"2024-09-21T02:07:27.805484+0000","last_clean":"2024-09-21T02:07:27.805484+0000","last_became_active":"2024-09-21T02:03:57.083918+0000","last_became_peered":"2024-09-21T02:03:57.083918+0000","last_unstale":"2024-09-21T02:07:27.805484+0000","last_undegraded":"2024-09-21T02:07:27.805484+0000","last_fullsized":"2024-09-21T02:07:27.805484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-09-21T02:07:27.805388+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-09-21T02:07:27.805388+0000","last_clean_scrub_stamp":"2024-09-21T02:07:27.805388+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:04.151675+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'154","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:27.229579+0000","last_change":"2024-09-21T02:07:27.229579+0000","last_active":"2024-09-21T02:07:27.229579+0000","last_peered":"2024-09-21T02:07:27.229579+0000","last_clean":"2024-09-21T02:07:27.229579+0000","last_became_active":"2024-09-21T02:03:58.079573+0000","last_became_peered":"2024-09-21T02:03:58.079573+0000","last_unstale":"2024-09-21T02:07:27.229579+0000","last_undegraded":"2024-09-21T02:07:27.229579+0000","last_fullsized":"2024-09-21T02:07:27.229579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'154","last_scrub_stamp":"2024-09-21T02:07:27.229470+0000","last_deep_scrub":"23'154","last_deep_scrub_stamp":"2024-09-21T02:07:27.229470+0000","last_clean_scrub_stamp":"2024-09-21T02:07:27.229470+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:23:46.211266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'205","reported_seq":231,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:29.325187+0000","last_change":"2024-09-21T02:07:29.325187+0000","last_active":"2024-09-21T02:07:29.325187+0000","last_peered":"2024-09-21T02:07:29.325187+0000","last_clean":"2024-09-21T02:07:29.325187+0000","last_became_active":"2024-09-21T02:03:58.066812+0000","last_became_peered":"2024-09-21T02:03:58.066812+0000","last_unstale":"2024-09-21T02:07:29.325187+0000","last_undegraded":"2024-09-21T02:07:29.325187+0000","last_fullsized":"2024-09-21T02:07:29.325187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'205","last_scrub_stamp":"2024-09-21T02:07:29.325097+0000","last_deep_scrub":"23'205","last_deep_scrub_stamp":"2024-09-21T02:07:29.325097+0000","last_clean_scrub_stamp":"2024-09-21T02:07:29.325097+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:25.914822+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'3","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.850229+0000","last_change":"2024-09-21T02:07:22.850229+0000","last_active":"2024-09-21T02:07:22.850229+0000","last_peered":"2024-09-21T02:07:22.850229+0000","last_clean":"2024-09-21T02:07:22.850229+0000","last_became_active":"2024-09-21T02:03:57.080770+0000","last_became_peered":"2024-09-21T02:03:57.080770+0000","last_unstale":"2024-09-21T02:07:22.850229+0000","last_undegraded":"2024-09-21T02:07:22.850229+0000","last_fullsized":"2024-09-21T02:07:22.850229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-09-21T02:07:22.850118+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-09-21T02:07:22.850118+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.850118+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.421894+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'162","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:28.334734+0000","last_change":"2024-09-21T02:07:28.334734+0000","last_active":"2024-09-21T02:07:28.334734+0000","last_peered":"2024-09-21T02:07:28.334734+0000","last_clean":"2024-09-21T02:07:28.334734+0000","last_became_active":"2024-09-21T02:03:58.081594+0000","last_became_peered":"2024-09-21T02:03:58.081594+0000","last_unstale":"2024-09-21T02:07:28.334734+0000","last_undegraded":"2024-09-21T02:07:28.334734+0000","last_fullsized":"2024-09-21T02:07:28.334734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'162","last_scrub_stamp":"2024-09-21T02:07:28.334606+0000","last_deep_scrub":"23'162","last_deep_scrub_stamp":"2024-09-21T02:07:28.334606+0000","last_clean_scrub_stamp":"2024-09-21T02:07:28.334606+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:49.585845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'8","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:25.854330+0000","last_change":"2024-09-21T02:07:25.854330+0000","last_active":"2024-09-21T02:07:25.854330+0000","last_peered":"2024-09-21T02:07:25.854330+0000","last_clean":"2024-09-21T02:07:25.854330+0000","last_became_active":"2024-09-21T02:03:57.082472+0000","last_became_peered":"2024-09-21T02:03:57.082472+0000","last_unstale":"2024-09-21T02:07:25.854330+0000","last_undegraded":"2024-09-21T02:07:25.854330+0000","last_fullsized":"2024-09-21T02:07:25.854330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-09-21T02:07:25.854232+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-09-21T02:07:25.854232+0000","last_clean_scrub_stamp":"2024-09-21T02:07:25.854232+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:50.262803+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'160","reported_seq":186,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:24.356328+0000","last_change":"2024-09-21T02:07:24.356328+0000","last_active":"2024-09-21T02:07:24.356328+0000","last_peered":"2024-09-21T02:07:24.356328+0000","last_clean":"2024-09-21T02:07:24.356328+0000","last_became_active":"2024-09-21T02:03:58.080068+0000","last_became_peered":"2024-09-21T02:03:58.080068+0000","last_unstale":"2024-09-21T02:07:24.356328+0000","last_undegraded":"2024-09-21T02:07:24.356328+0000","last_fullsized":"2024-09-21T02:07:24.356328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'160","last_scrub_stamp":"2024-09-21T02:07:24.356229+0000","last_deep_scrub":"23'160","last_deep_scrub_stamp":"2024-09-21T02:07:24.356229+0000","last_clean_scrub_stamp":"2024-09-21T02:07:24.356229+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:21.418442+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:22.478796+0000","last_change":"2024-09-21T02:07:22.478796+0000","last_active":"2024-09-21T02:07:22.478796+0000","last_peered":"2024-09-21T02:07:22.478796+0000","last_clean":"2024-09-21T02:07:22.478796+0000","last_became_active":"2024-09-21T02:03:57.079209+0000","last_became_peered":"2024-09-21T02:03:57.079209+0000","last_unstale":"2024-09-21T02:07:22.478796+0000","last_undegraded":"2024-09-21T02:07:22.478796+0000","last_fullsized":"2024-09-21T02:07:22.478796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2024-09-21T02:07:22.478670+0000","last_deep_scrub":"21'2","last_deep_scrub_stamp":"2024-09-21T02:07:22.478670+0000","last_clean_scrub_stamp":"2024-09-21T02:07:22.478670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:50:09.049812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'181","reported_seq":207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:28.471788+0000","last_change":"2024-09-21T02:07:28.471788+0000","last_active":"2024-09-21T02:07:28.471788+0000","last_peered":"2024-09-21T02:07:28.471788+0000","last_clean":"2024-09-21T02:07:28.471788+0000","last_became_active":"2024-09-21T02:03:58.079065+0000","last_became_peered":"2024-09-21T02:03:58.079065+0000","last_unstale":"2024-09-21T02:07:28.471788+0000","last_undegraded":"2024-09-21T02:07:28.471788+0000","last_fullsized":"2024-09-21T02:07:28.471788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'181","last_scrub_stamp":"2024-09-21T02:07:28.471685+0000","last_deep_scrub":"23'181","last_deep_scrub_stamp":"2024-09-21T02:07:28.471685+0000","last_clean_scrub_stamp":"2024-09-21T02:07:28.471685+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:44.939565+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'13","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:07:19.285919+0000","last_change":"2024-09-21T02:07:19.285919+0000","last_active":"2024-09-21T02:07:19.285919+0000","last_peered":"2024-09-21T02:07:19.285919+0000","last_clean":"2024-09-21T02:07:19.285919+0000","last_became_active":"2024-09-21T02:03:57.069330+0000","last_became_peered":"2024-09-21T02:03:57.069330+0000","last_unstale":"2024-09-21T02:07:19.285919+0000","last_undegraded":"2024-09-21T02:07:19.285919+0000","last_fullsized":"2024-09-21T02:07:19.285919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13","last_scrub_stamp":"2024-09-21T02:07:19.285838+0000","last_deep_scrub":"23'13","last_deep_scrub_stamp":"2024-09-21T02:07:19.285838+0000","last_clean_scrub_stamp":"2024-09-21T02:07:19.285838+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:00.264480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11255,"ondisk_log_size":11255,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7623955,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3826,"num_write_kb":11018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15409152,"data_stored":15247910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3856,"ondisk_log_size":3856,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57568,"kb_used_data":4176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314272,"statfs":{"total":96636764160,"available":96577814528,"internally_reserved":0,"allocated":4276224,"data_stored":4101088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":86315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":86048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83232,"kb_used_data":7696,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94288608,"statfs":{"total":96636764160,"available":96551534592,"internally_reserved":0,"allocated":7880704,"data_stored":7705597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":86260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":258048,"data_stored":85881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53572,"kb_used_data":4340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318268,"statfs":{"total":96636764160,"available":96581906432,"internally_reserved":0,"allocated":4444160,"data_stored":4280988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":843776,"data_stored":675965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7647232,"data_stored":7620660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4196051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3448832,"data_stored":3425783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:07:42.979 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T02:07:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 -- 172.21.15.88:0/3393769757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 msgr2=0x7f54980cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 --2- 172.21.15.88:0/3393769757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f54980cf470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5490005fd0 tx=0x7f549002a930 comp rx=0 tx=0).stop 2024-09-21T02:07:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 -- 172.21.15.88:0/3393769757 shutdown_connections 2024-09-21T02:07:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 --2- 172.21.15.88:0/3393769757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f54980cf470 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 --2- 172.21.15.88:0/3393769757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5498153a70 0x7f54980cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 --2- 172.21.15.88:0/3393769757 >> v2:172.21.15.195:3300/0 conn(0x7f5498068d00 0x7f549815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 -- 172.21.15.88:0/3393769757 >> 172.21.15.88:0/3393769757 conn(0x7f5498070960 msgr2=0x7f549806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 -- 172.21.15.88:0/3393769757 shutdown_connections 2024-09-21T02:07:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.125+0000 7f549c982640 1 -- 172.21.15.88:0/3393769757 wait complete. 2024-09-21T02:07:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.127+0000 7f549c982640 1 Processor -- start 2024-09-21T02:07:43.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.127+0000 7f549c982640 1 -- start start 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f549c982640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5498068d00 0x7f54981efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f549c982640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f549815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f549c982640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5498153a70 0x7f549814f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f549c982640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5498159ed0 con 0x7f54980693a0 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f549c982640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5498159bf0 con 0x7f5498153a70 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f549c982640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f549815a6d0 con 0x7f5498068d00 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f549815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f549815d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48210/0 (socket says 172.21.15.88:48210) 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 -- 172.21.15.88:0/247540684 learned_addr learned my addr 172.21.15.88:0/247540684 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f549659c640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5498068d00 0x7f54981efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f5496d9d640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5498153a70 0x7f549814f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 -- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5498068d00 msgr2=0x7f54981efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5498068d00 0x7f54981efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 -- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5498153a70 msgr2=0x7f549814f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5498153a70 0x7f549814f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 -- 172.21.15.88:0/247540684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54981f3460 con 0x7f54980693a0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548dd9b640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f549815d240 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f548000d700 tx=0x7f548000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.129+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480017070 con 0x7f54980693a0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.130+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5480009b10 con 0x7f54980693a0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.130+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480010460 con 0x7f54980693a0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.130+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54981f0390 con 0x7f54980693a0 2024-09-21T02:07:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.130+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54981f39b0 con 0x7f54980693a0 2024-09-21T02:07:43.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.131+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54980c7110 con 0x7f54980693a0 2024-09-21T02:07:43.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.131+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f548001a070 con 0x7f54980693a0 2024-09-21T02:07:43.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.132+0000 7f548f7fe640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5478078bc0 0x7f547807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.132+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f548009ad60 con 0x7f54980693a0 2024-09-21T02:07:43.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.132+0000 7f549659c640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5478078bc0 0x7f547807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.133+0000 7f549659c640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5478078bc0 0x7f547807b0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5488005fd0 tx=0x7f548800b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:43.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.137+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5480062bf0 con 0x7f54980693a0 2024-09-21T02:07:43.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.314+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f54980cfc00 con 0x7f54980693a0 2024-09-21T02:07:43.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.318+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v37)=0 v37) ==== 123+0+0 (secure 0 0 0) 0x7f5480067620 con 0x7f54980693a0 2024-09-21T02:07:43.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.320+0000 7f548f7fe640 1 -- 172.21.15.88:0/247540684 <== mon.0 v2:172.21.15.88:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f5480062700 con 0x7f54980693a0 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5478078bc0 msgr2=0x7f547807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5478078bc0 0x7f547807b0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5488005fd0 tx=0x7f548800b260 comp rx=0 tx=0).stop 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 msgr2=0x7f549815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f549815d240 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f548000d700 tx=0x7f548000dbc0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 shutdown_connections 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5478078bc0 0x7f547807b0d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5498153a70 0x7f549814f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54980693a0 0x7f549815d240 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 --2- 172.21.15.88:0/247540684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5498068d00 0x7f54981efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.321+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 >> 172.21.15.88:0/247540684 conn(0x7f5498070960 msgr2=0x7f5498072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:43.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.322+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 shutdown_connections 2024-09-21T02:07:43.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.322+0000 7f549c982640 1 -- 172.21.15.88:0/247540684 wait complete. 2024-09-21T02:07:43.334 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:07:43.334 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:07:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.445+0000 7f2d466dd640 1 -- 172.21.15.88:0/4218578345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d28004800 msgr2=0x7f2d28004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.445+0000 7f2d466dd640 1 --2- 172.21.15.88:0/4218578345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d28004800 0x7f2d28004c10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009b50 tx=0x7f2d28005830 comp rx=0 tx=0).stop 2024-09-21T02:07:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.445+0000 7f2d44c7a640 1 -- 172.21.15.88:0/4218578345 reap_dead start 2024-09-21T02:07:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.445+0000 7f2d466dd640 1 -- 172.21.15.88:0/4218578345 shutdown_connections 2024-09-21T02:07:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.445+0000 7f2d466dd640 1 -- 172.21.15.88:0/4218578345 >> 172.21.15.88:0/4218578345 conn(0x7f2d400729d0 msgr2=0x7f2d40072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.445+0000 7f2d466dd640 1 -- 172.21.15.88:0/4218578345 shutdown_connections 2024-09-21T02:07:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.445+0000 7f2d466dd640 1 -- 172.21.15.88:0/4218578345 wait complete. 2024-09-21T02:07:43.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.447+0000 7f2d466dd640 1 Processor -- start 2024-09-21T02:07:43.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.447+0000 7f2d466dd640 1 -- start start 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.448+0000 7f2d466dd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28004800 0x7f2d4006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.448+0000 7f2d466dd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d400cf090 0x7f2d401e2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.448+0000 7f2d466dd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d401e2770 0x7f2d4014a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d466dd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4015a5b0 con 0x7f2d401e2770 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d466dd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d40070ab0 con 0x7f2d28004800 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d466dd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f2d400c8f30 con 0x7f2d400cf090 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d401e2770 0x7f2d4014a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d401e2770 0x7f2d4014a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48228/0 (socket says 172.21.15.88:48228) 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 -- 172.21.15.88:0/599672073 learned_addr learned my addr 172.21.15.88:0/599672073 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d3ffff640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28004800 0x7f2d4006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d3f7fe640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d400cf090 0x7f2d401e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 -- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d400cf090 msgr2=0x7f2d401e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d400cf090 0x7f2d401e2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 -- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28004800 msgr2=0x7f2d4006e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28004800 0x7f2d4006e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 -- 172.21.15.88:0/599672073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4006e650 con 0x7f2d401e2770 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d44c7a640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d401e2770 0x7f2d4014a6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2d38002230 tx=0x7f2d380066d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d3cff9640 1 -- 172.21.15.88:0/599672073 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800fa10 con 0x7f2d401e2770 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d3cff9640 1 -- 172.21.15.88:0/599672073 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f2d38016020 con 0x7f2d401e2770 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d3cff9640 1 -- 172.21.15.88:0/599672073 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38011430 con 0x7f2d401e2770 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d401e0d80 con 0x7f2d401e2770 2024-09-21T02:07:43.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.449+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d401e1340 con 0x7f2d401e2770 2024-09-21T02:07:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.451+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d40156df0 con 0x7f2d401e2770 2024-09-21T02:07:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.451+0000 7f2d3cff9640 1 -- 172.21.15.88:0/599672073 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d380115d0 con 0x7f2d401e2770 2024-09-21T02:07:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.451+0000 7f2d3cff9640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d2c078b90 0x7f2d2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.452+0000 7f2d3cff9640 1 -- 172.21.15.88:0/599672073 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2d3809b760 con 0x7f2d401e2770 2024-09-21T02:07:43.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.452+0000 7f2d3ffff640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d2c078b90 0x7f2d2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.452+0000 7f2d3ffff640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d2c078b90 0x7f2d2c07b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009bd0 tx=0x7f2d30005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:43.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.456+0000 7f2d3cff9640 1 -- 172.21.15.88:0/599672073 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d38020030 con 0x7f2d401e2770 2024-09-21T02:07:43.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.672+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d40072360 con 0x7f2d401e2770 2024-09-21T02:07:43.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.673+0000 7f2d3cff9640 1 -- 172.21.15.88:0/599672073 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6908 (secure 0 0 0) 0x7f2d3806a030 con 0x7f2d401e2770 2024-09-21T02:07:43.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:43.674 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":13,"btime":"2024-09-21T02:04:16:141092+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:16.141080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4398,4399,4401]},"id":1}]} 2024-09-21T02:07:43.674 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 13 2024-09-21T02:07:43.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d2c078b90 msgr2=0x7f2d2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d2c078b90 0x7f2d2c07b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009bd0 tx=0x7f2d30005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:43.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d401e2770 msgr2=0x7f2d4014a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d401e2770 0x7f2d4014a6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2d38002230 tx=0x7f2d380066d0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 shutdown_connections 2024-09-21T02:07:43.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f2d2c078b90 0x7f2d2c07b0a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d401e2770 0x7f2d4014a6e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d400cf090 0x7f2d401e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 --2- 172.21.15.88:0/599672073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d28004800 0x7f2d4006e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 >> 172.21.15.88:0/599672073 conn(0x7f2d400729d0 msgr2=0x7f2d400c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:43.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 shutdown_connections 2024-09-21T02:07:43.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.675+0000 7f2d466dd640 1 -- 172.21.15.88:0/599672073 wait complete. 2024-09-21T02:07:43.688 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:07:43.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 -- 172.21.15.88:0/627296776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940ccd00 msgr2=0x7f529415f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 --2- 172.21.15.88:0/627296776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940ccd00 0x7f529415f970 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5284009830 tx=0x7f528402a5a0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 -- 172.21.15.88:0/627296776 shutdown_connections 2024-09-21T02:07:43.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 --2- 172.21.15.88:0/627296776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52940cad60 0x7f529406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 --2- 172.21.15.88:0/627296776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940ccd00 0x7f529415f970 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 -- 172.21.15.88:0/627296776 >> 172.21.15.88:0/627296776 conn(0x7f52940cd9f0 msgr2=0x7f52940cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:43.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 -- 172.21.15.88:0/627296776 shutdown_connections 2024-09-21T02:07:43.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.801+0000 7f5299430640 1 -- 172.21.15.88:0/627296776 wait complete. 2024-09-21T02:07:43.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.802+0000 7f5299430640 1 Processor -- start 2024-09-21T02:07:43.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.803+0000 7f5299430640 1 -- start start 2024-09-21T02:07:43.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5299430640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52940cad60 0x7f52941f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5299430640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940cb170 0x7f529415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5299430640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52940ccd00 0x7f5294158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5299430640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f529406dcd0 con 0x7f52940cad60 2024-09-21T02:07:43.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5299430640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f529415c640 con 0x7f52940cb170 2024-09-21T02:07:43.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5299430640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f529406de50 con 0x7f52940ccd00 2024-09-21T02:07:43.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52940cad60 0x7f52941f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52940cad60 0x7f52941f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48242/0 (socket says 172.21.15.88:48242) 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 -- 172.21.15.88:0/2164134493 learned_addr learned my addr 172.21.15.88:0/2164134493 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f528a7fc640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940cb170 0x7f529415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f52937fe640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52940ccd00 0x7f5294158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 -- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52940ccd00 msgr2=0x7f5294158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52940ccd00 0x7f5294158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 -- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940cb170 msgr2=0x7f529415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940cb170 0x7f529415e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.804+0000 7f5292ffd640 1 -- 172.21.15.88:0/2164134493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52941599b0 con 0x7f52940cad60 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.805+0000 7f5292ffd640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52940cad60 0x7f52941f2d60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5284009920 tx=0x7f52840052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.805+0000 7f528bfff640 1 -- 172.21.15.88:0/2164134493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284005c90 con 0x7f52940cad60 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.805+0000 7f528bfff640 1 -- 172.21.15.88:0/2164134493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f5284031d70 con 0x7f52940cad60 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.805+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52941596d0 con 0x7f52940cad60 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.805+0000 7f528bfff640 1 -- 172.21.15.88:0/2164134493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284004990 con 0x7f52940cad60 2024-09-21T02:07:43.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.805+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52941f2560 con 0x7f52940cad60 2024-09-21T02:07:43.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.806+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52940c8dd0 con 0x7f52940cad60 2024-09-21T02:07:43.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.807+0000 7f528bfff640 1 -- 172.21.15.88:0/2164134493 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5284031750 con 0x7f52940cad60 2024-09-21T02:07:43.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.807+0000 7f528bfff640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5274078bc0 0x7f527407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:43.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.807+0000 7f528bfff640 1 -- 172.21.15.88:0/2164134493 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f52840bc040 con 0x7f52940cad60 2024-09-21T02:07:43.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.807+0000 7f528a7fc640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5274078bc0 0x7f527407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:43.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.808+0000 7f528a7fc640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5274078bc0 0x7f527407b0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f529408e650 tx=0x7f527c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:43.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:43.812+0000 7f528bfff640 1 -- 172.21.15.88:0/2164134493 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f52940c8dd0 con 0x7f52940cad60 2024-09-21T02:07:44.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.025+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f529415f970 con 0x7f52940cad60 2024-09-21T02:07:44.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.025+0000 7f528bfff640 1 -- 172.21.15.88:0/2164134493 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6908 (secure 0 0 0) 0x7f528403f030 con 0x7f52940cad60 2024-09-21T02:07:44.027 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:44.027 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":13,"btime":"2024-09-21T02:04:16:141092+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/632988575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":632988575},{"type":"v1","addr":"172.21.15.195:6835","nonce":632988575}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1742361553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1742361553},{"type":"v1","addr":"172.21.15.88:6834","nonce":1742361553}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/510649169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":510649169},{"type":"v1","addr":"172.21.15.88:6835","nonce":510649169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:03:57.430092+0000","modified":"2024-09-21T02:04:16.141080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4401,"mds_1":4399,"mds_2":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"f","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.195:6837/778249012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":778249012},{"type":"v1","addr":"172.21.15.195:6837","nonce":778249012}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4399":{"gid":4399,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6837/3330183943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":3330183943},{"type":"v1","addr":"172.21.15.88:6837","nonce":3330183943}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4401":{"gid":4401,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.195:6839/3087611678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":3087611678},{"type":"v1","addr":"172.21.15.195:6839","nonce":3087611678}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4401,"qdb_cluster":[4398,4399,4401]},"id":1}]} 2024-09-21T02:07:44.027 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 13 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.027+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5274078bc0 msgr2=0x7f527407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.027+0000 7f5299430640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5274078bc0 0x7f527407b0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f529408e650 tx=0x7f527c005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52940cad60 msgr2=0x7f52941f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52940cad60 0x7f52941f2d60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5284009920 tx=0x7f52840052f0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 shutdown_connections 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f5274078bc0 0x7f527407b0d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52940ccd00 0x7f5294158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52940cb170 0x7f529415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 --2- 172.21.15.88:0/2164134493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52940cad60 0x7f52941f2d60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 >> 172.21.15.88:0/2164134493 conn(0x7f52940cd9f0 msgr2=0x7f52941e6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:44.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 shutdown_connections 2024-09-21T02:07:44.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.028+0000 7f5299430640 1 -- 172.21.15.88:0/2164134493 wait complete. 2024-09-21T02:07:44.042 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:07:44.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.157+0000 7f31176f2640 1 -- 172.21.15.88:0/2123321366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31100b2d90 msgr2=0x7f31100c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.157+0000 7f31176f2640 1 --2- 172.21.15.88:0/2123321366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31100b2d90 0x7f31100c2140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f310400a940 tx=0x7f310402b980 comp rx=0 tx=0).stop 2024-09-21T02:07:44.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.157+0000 7f31176f2640 1 -- 172.21.15.88:0/2123321366 shutdown_connections 2024-09-21T02:07:44.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.157+0000 7f31176f2640 1 --2- 172.21.15.88:0/2123321366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31100b2d90 0x7f31100c2140 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.157+0000 7f31176f2640 1 -- 172.21.15.88:0/2123321366 >> 172.21.15.88:0/2123321366 conn(0x7f3110070c30 msgr2=0x7f3110071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:44.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.157+0000 7f31176f2640 1 -- 172.21.15.88:0/2123321366 shutdown_connections 2024-09-21T02:07:44.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.157+0000 7f31176f2640 1 -- 172.21.15.88:0/2123321366 wait complete. 2024-09-21T02:07:44.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.158+0000 7f31176f2640 1 Processor -- start 2024-09-21T02:07:44.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.159+0000 7f31176f2640 1 -- start start 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f31176f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311006cd60 0x7f31101f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f31176f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31100b2d90 0x7f3110147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f31176f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3110151ec0 0x7f31100c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f31176f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31100b31a0 con 0x7f3110151ec0 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f31176f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f31100c7870 con 0x7f31100b2d90 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f31176f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f31100c2680 con 0x7f311006cd60 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3110151ec0 0x7f31100c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3110151ec0 0x7f31100c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48264/0 (socket says 172.21.15.88:48264) 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 -- 172.21.15.88:0/3319898562 learned_addr learned my addr 172.21.15.88:0/3319898562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:44.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3114c8d640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31100b2d90 0x7f3110147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f311548e640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311006cd60 0x7f31101f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 -- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311006cd60 msgr2=0x7f31101f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311006cd60 0x7f31101f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 -- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31100b2d90 msgr2=0x7f3110147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31100b2d90 0x7f3110147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 -- 172.21.15.88:0/3319898562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31101f2610 con 0x7f3110151ec0 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.160+0000 7f3115c8f640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3110151ec0 0x7f31100c5930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f310800cb40 tx=0x7f310800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.161+0000 7f3101ffb640 1 -- 172.21.15.88:0/3319898562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108016020 con 0x7f3110151ec0 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.161+0000 7f3101ffb640 1 -- 172.21.15.88:0/3319898562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f310800ac40 con 0x7f3110151ec0 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.161+0000 7f3101ffb640 1 -- 172.21.15.88:0/3319898562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108012680 con 0x7f3110151ec0 2024-09-21T02:07:44.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.161+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31101f28d0 con 0x7f3110151ec0 2024-09-21T02:07:44.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.161+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31101f2e90 con 0x7f3110151ec0 2024-09-21T02:07:44.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.162+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3110073920 con 0x7f3110151ec0 2024-09-21T02:07:44.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.162+0000 7f3101ffb640 1 -- 172.21.15.88:0/3319898562 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3108019020 con 0x7f3110151ec0 2024-09-21T02:07:44.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.163+0000 7f3101ffb640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f30f4078bc0 0x7f30f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.163+0000 7f3101ffb640 1 -- 172.21.15.88:0/3319898562 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f310809b100 con 0x7f3110151ec0 2024-09-21T02:07:44.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.163+0000 7f311548e640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f30f4078bc0 0x7f30f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.164+0000 7f311548e640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f30f4078bc0 0x7f30f407b0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f310c0015e0 tx=0x7f310c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:44.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.168+0000 7f3101ffb640 1 -- 172.21.15.88:0/3319898562 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3110073920 con 0x7f3110151ec0 2024-09-21T02:07:44.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.350+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3110155a60 con 0x7f3110151ec0 2024-09-21T02:07:44.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.352+0000 7f3101ffb640 1 -- 172.21.15.88:0/3319898562 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16915 (secure 0 0 0) 0x7f3110155a60 con 0x7f3110151ec0 2024-09-21T02:07:44.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T02:07:44.353 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"0ee3c2c3-8406-48b8-a5fb-ec622d976411","created":"2024-09-21T02:03:32.503482+0000","modified":"2024-09-21T02:04:45.193137+0000","last_up_change":"2024-09-21T02:03:45.801953+0000","last_in_change":"2024-09-21T02:03:36.330786+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:03:44.834536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:03:55.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:03:56.381370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c94ead8-72ba-41a1-b18a-c9b2c1de32c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6801","nonce":1826444869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6803","nonce":1826444869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6807","nonce":1826444869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":1826444869},{"type":"v1","addr":"172.21.15.88:6805","nonce":1826444869}]},"public_addr":"172.21.15.88:6801/1826444869","cluster_addr":"172.21.15.88:6803/1826444869","heartbeat_back_addr":"172.21.15.88:6807/1826444869","heartbeat_front_addr":"172.21.15.88:6805/1826444869","state":["exists","up"]},{"osd":1,"uuid":"dd02029f-64ef-4c29-b948-68face049bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6825","nonce":2879208543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6827","nonce":2879208543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6831","nonce":2879208543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2879208543},{"type":"v1","addr":"172.21.15.88:6829","nonce":2879208543}]},"public_addr":"172.21.15.88:6825/2879208543","cluster_addr":"172.21.15.88:6827/2879208543","heartbeat_back_addr":"172.21.15.88:6831/2879208543","heartbeat_front_addr":"172.21.15.88:6829/2879208543","state":["exists","up"]},{"osd":2,"uuid":"f0c10054-48b6-4b37-ac91-1501da5ef01f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6809","nonce":1554825340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6811","nonce":1554825340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6815","nonce":1554825340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1554825340},{"type":"v1","addr":"172.21.15.88:6813","nonce":1554825340}]},"public_addr":"172.21.15.88:6809/1554825340","cluster_addr":"172.21.15.88:6811/1554825340","heartbeat_back_addr":"172.21.15.88:6815/1554825340","heartbeat_front_addr":"172.21.15.88:6813/1554825340","state":["exists","up"]},{"osd":3,"uuid":"2f19c3d6-3682-430a-9c0c-9bbe0d92d338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6817","nonce":723368200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6819","nonce":723368200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6823","nonce":723368200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":723368200},{"type":"v1","addr":"172.21.15.88:6821","nonce":723368200}]},"public_addr":"172.21.15.88:6817/723368200","cluster_addr":"172.21.15.88:6819/723368200","heartbeat_back_addr":"172.21.15.88:6823/723368200","heartbeat_front_addr":"172.21.15.88:6821/723368200","state":["exists","up"]},{"osd":4,"uuid":"5e972a73-17cc-4107-9991-07c6ef899ddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6819","nonce":2209751664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6821","nonce":2209751664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6825","nonce":2209751664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2209751664},{"type":"v1","addr":"172.21.15.195:6823","nonce":2209751664}]},"public_addr":"172.21.15.195:6819/2209751664","cluster_addr":"172.21.15.195:6821/2209751664","heartbeat_back_addr":"172.21.15.195:6825/2209751664","heartbeat_front_addr":"172.21.15.195:6823/2209751664","state":["exists","up"]},{"osd":5,"uuid":"6a006ffa-9f45-4aca-a9dc-a6ef17e3b89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6827","nonce":3321376084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6829","nonce":3321376084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6833","nonce":3321376084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3321376084},{"type":"v1","addr":"172.21.15.195:6831","nonce":3321376084}]},"public_addr":"172.21.15.195:6827/3321376084","cluster_addr":"172.21.15.195:6829/3321376084","heartbeat_back_addr":"172.21.15.195:6833/3321376084","heartbeat_front_addr":"172.21.15.195:6831/3321376084","state":["exists","up"]},{"osd":6,"uuid":"f048a1c4-2584-43fd-b35d-cc9f5297f892","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6803","nonce":1905698167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6805","nonce":1905698167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6809","nonce":1905698167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1905698167},{"type":"v1","addr":"172.21.15.195:6807","nonce":1905698167}]},"public_addr":"172.21.15.195:6803/1905698167","cluster_addr":"172.21.15.195:6805/1905698167","heartbeat_back_addr":"172.21.15.195:6809/1905698167","heartbeat_front_addr":"172.21.15.195:6807/1905698167","state":["exists","up"]},{"osd":7,"uuid":"8dc8088b-3941-476b-8d8b-e946018d3842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6811","nonce":985762803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6813","nonce":985762803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6817","nonce":985762803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":985762803},{"type":"v1","addr":"172.21.15.195:6815","nonce":985762803}]},"public_addr":"172.21.15.195:6811/985762803","cluster_addr":"172.21.15.195:6813/985762803","heartbeat_back_addr":"172.21.15.195:6817/985762803","heartbeat_front_addr":"172.21.15.195:6815/985762803","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.328372+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.322931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.899092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.113823+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.660141+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:43.193660+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.419567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:03:42.796032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f30f4078bc0 msgr2=0x7f30f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f30f4078bc0 0x7f30f407b0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f310c0015e0 tx=0x7f310c009040 comp rx=0 tx=0).stop 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3110151ec0 msgr2=0x7f31100c5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3110151ec0 0x7f31100c5930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f310800cb40 tx=0x7f310800a060 comp rx=0 tx=0).stop 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 shutdown_connections 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f30f4078bc0 0x7f30f407b0d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3110151ec0 0x7f31100c5930 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31100b2d90 0x7f3110147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 --2- 172.21.15.88:0/3319898562 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311006cd60 0x7f31101f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 >> 172.21.15.88:0/3319898562 conn(0x7f3110070c30 msgr2=0x7f311014a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 shutdown_connections 2024-09-21T02:07:44.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.354+0000 7f31176f2640 1 -- 172.21.15.88:0/3319898562 wait complete. 2024-09-21T02:07:44.367 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:07:44.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.483+0000 7efd58ac5640 1 -- 172.21.15.88:0/1250640485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 msgr2=0x7efd3c004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.483+0000 7efd58ac5640 1 --2- 172.21.15.88:0/1250640485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 0x7efd3c004060 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efd44006fb0 tx=0x7efd3c005830 comp rx=0 tx=0).stop 2024-09-21T02:07:44.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.483+0000 7efd52d9d640 1 -- 172.21.15.88:0/1250640485 reap_dead start 2024-09-21T02:07:44.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.483+0000 7efd58ac5640 1 -- 172.21.15.88:0/1250640485 shutdown_connections 2024-09-21T02:07:44.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.483+0000 7efd58ac5640 1 -- 172.21.15.88:0/1250640485 >> 172.21.15.88:0/1250640485 conn(0x7efd540cda90 msgr2=0x7efd540cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:44.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.483+0000 7efd58ac5640 1 -- 172.21.15.88:0/1250640485 shutdown_connections 2024-09-21T02:07:44.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.483+0000 7efd58ac5640 1 -- 172.21.15.88:0/1250640485 wait complete. 2024-09-21T02:07:44.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.485+0000 7efd58ac5640 1 Processor -- start 2024-09-21T02:07:44.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.485+0000 7efd58ac5640 1 -- start start 2024-09-21T02:07:44.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd58ac5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 0x7efd541f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd58ac5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540ca4d0 0x7efd540c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd58ac5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd541f21a0 0x7efd541f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd58ac5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd540b4410 con 0x7efd3c003c10 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd58ac5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efd540b4130 con 0x7efd540ca4d0 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd58ac5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7efd540b46f0 con 0x7efd541f21a0 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd5259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 0x7efd541f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd5259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 0x7efd541f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48288/0 (socket says 172.21.15.88:48288) 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd5259c640 1 -- 172.21.15.88:0/755402409 learned_addr learned my addr 172.21.15.88:0/755402409 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.486+0000 7efd51d9b640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540ca4d0 0x7efd540c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd52d9d640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd541f21a0 0x7efd541f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd5259c640 1 -- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd541f21a0 msgr2=0x7efd541f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd5259c640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd541f21a0 0x7efd541f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd5259c640 1 -- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540ca4d0 msgr2=0x7efd540c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd5259c640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540ca4d0 0x7efd540c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd5259c640 1 -- 172.21.15.88:0/755402409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd541f5ca0 con 0x7efd3c003c10 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd5259c640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 0x7efd541f5760 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd44031040 tx=0x7efd44006c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd42ffd640 1 -- 172.21.15.88:0/755402409 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44032030 con 0x7efd3c003c10 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd42ffd640 1 -- 172.21.15.88:0/755402409 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7efd44003b10 con 0x7efd3c003c10 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd42ffd640 1 -- 172.21.15.88:0/755402409 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4402e480 con 0x7efd3c003c10 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd541f1ab0 con 0x7efd3c003c10 2024-09-21T02:07:44.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.487+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd540c7730 con 0x7efd3c003c10 2024-09-21T02:07:44.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.489+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd540b3e80 con 0x7efd3c003c10 2024-09-21T02:07:44.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.489+0000 7efd42ffd640 1 -- 172.21.15.88:0/755402409 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd44035020 con 0x7efd3c003c10 2024-09-21T02:07:44.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.490+0000 7efd42ffd640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efd34078b90 0x7efd3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.490+0000 7efd42ffd640 1 -- 172.21.15.88:0/755402409 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efd440bb560 con 0x7efd3c003c10 2024-09-21T02:07:44.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.490+0000 7efd51d9b640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efd34078b90 0x7efd3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.491+0000 7efd51d9b640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efd34078b90 0x7efd3407b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd5415b140 tx=0x7efd48005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:44.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.494+0000 7efd42ffd640 1 -- 172.21.15.88:0/755402409 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd44083420 con 0x7efd3c003c10 2024-09-21T02:07:44.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.715+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efd540c74e0 con 0x7efd3c003c10 2024-09-21T02:07:44.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.717+0000 7efd42ffd640 1 -- 172.21.15.88:0/755402409 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) ==== 161+0+0 (secure 0 0 0) 0x7efd44087e50 con 0x7efd3c003c10 2024-09-21T02:07:44.718 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:07:44.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efd34078b90 msgr2=0x7efd3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efd34078b90 0x7efd3407b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd5415b140 tx=0x7efd48005f90 comp rx=0 tx=0).stop 2024-09-21T02:07:44.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 msgr2=0x7efd541f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 0x7efd541f5760 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd44031040 tx=0x7efd44006c60 comp rx=0 tx=0).stop 2024-09-21T02:07:44.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 shutdown_connections 2024-09-21T02:07:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7efd34078b90 0x7efd3407b0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd541f21a0 0x7efd541f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540ca4d0 0x7efd540c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 --2- 172.21.15.88:0/755402409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd3c003c10 0x7efd541f5760 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 >> 172.21.15.88:0/755402409 conn(0x7efd540cda90 msgr2=0x7efd540cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 shutdown_connections 2024-09-21T02:07:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.720+0000 7efd58ac5640 1 -- 172.21.15.88:0/755402409 wait complete. 2024-09-21T02:07:44.734 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:07:44.781 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T02:07:44.788 INFO:tasks.ceph.mds.f.smithi195.stdout:starting mds.f at 2024-09-21T02:07:44.790 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2024-09-21T02:07:44.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.853+0000 7f7d85607640 1 -- 172.21.15.88:0/3854509584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 msgr2=0x7f7d8006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.853+0000 7f7d85607640 1 --2- 172.21.15.88:0/3854509584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 0x7f7d8006e280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009830 tx=0x7f7d7002a640 comp rx=0 tx=0).stop 2024-09-21T02:07:44.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.853+0000 7f7d85607640 1 -- 172.21.15.88:0/3854509584 shutdown_connections 2024-09-21T02:07:44.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.853+0000 7f7d85607640 1 --2- 172.21.15.88:0/3854509584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 0x7f7d8006e280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.853+0000 7f7d85607640 1 -- 172.21.15.88:0/3854509584 >> 172.21.15.88:0/3854509584 conn(0x7f7d800720b0 msgr2=0x7f7d800cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:44.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.853+0000 7f7d85607640 1 -- 172.21.15.88:0/3854509584 shutdown_connections 2024-09-21T02:07:44.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.853+0000 7f7d85607640 1 -- 172.21.15.88:0/3854509584 wait complete. 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.854+0000 7f7d85607640 1 Processor -- start 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.855+0000 7f7d85607640 1 -- start start 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d85607640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 0x7f7d801f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d85607640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d800c2e60 0x7f7d8014d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d85607640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c7af0 0x7f7d80150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d85607640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80159190 con 0x7f7d800c2e60 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d85607640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800c4930 con 0x7f7d8006de70 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d85607640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8006f1f0 con 0x7f7d800c7af0 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d800c2e60 0x7f7d8014d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d800c2e60 0x7f7d8014d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48306/0 (socket says 172.21.15.88:48306) 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 -- 172.21.15.88:0/3888029778 learned_addr learned my addr 172.21.15.88:0/3888029778 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:44.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7effd640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 0x7f7d801f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7f7fe640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c7af0 0x7f7d80150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 -- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c7af0 msgr2=0x7f7d80150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c7af0 0x7f7d80150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 -- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 msgr2=0x7f7d801f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 0x7f7d801f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 -- 172.21.15.88:0/3888029778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8015b9f0 con 0x7f7d800c2e60 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7effd640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 0x7f7d801f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d7e7fc640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d800c2e60 0x7f7d8014d9b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c800 tx=0x7f7d7400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d677fe640 1 -- 172.21.15.88:0/3888029778 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74017070 con 0x7f7d800c2e60 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d677fe640 1 -- 172.21.15.88:0/3888029778 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f7d74003da0 con 0x7f7d800c2e60 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8014def0 con 0x7f7d800c2e60 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.856+0000 7f7d677fe640 1 -- 172.21.15.88:0/3888029778 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740052d0 con 0x7f7d800c2e60 2024-09-21T02:07:44.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.857+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d801e25f0 con 0x7f7d800c2e60 2024-09-21T02:07:44.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.857+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d80152900 con 0x7f7d800c2e60 2024-09-21T02:07:44.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.858+0000 7f7d677fe640 1 -- 172.21.15.88:0/3888029778 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d7401a070 con 0x7f7d800c2e60 2024-09-21T02:07:44.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.859+0000 7f7d677fe640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7d6c078bc0 0x7f7d6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:44.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.859+0000 7f7d677fe640 1 -- 172.21.15.88:0/3888029778 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6083+0+0 (secure 0 0 0) 0x7f7d7409a480 con 0x7f7d800c2e60 2024-09-21T02:07:44.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.859+0000 7f7d7effd640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7d6c078bc0 0x7f7d6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:44.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.859+0000 7f7d7effd640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7d6c078bc0 0x7f7d6c07b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7d780013f0 tx=0x7f7d78009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:44.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:44.863+0000 7f7d677fe640 1 -- 172.21.15.88:0/3888029778 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d74062860 con 0x7f7d800c2e60 2024-09-21T02:07:45.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.078+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7d800cad20 con 0x7f7d800c2e60 2024-09-21T02:07:45.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.724+0000 7f7d677fe640 1 -- 172.21.15.88:0/3888029778 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f7d74062160 con 0x7f7d800c2e60 2024-09-21T02:07:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7d6c078bc0 msgr2=0x7f7d6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7d6c078bc0 0x7f7d6c07b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7d780013f0 tx=0x7f7d78009040 comp rx=0 tx=0).stop 2024-09-21T02:07:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d800c2e60 msgr2=0x7f7d8014d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d800c2e60 0x7f7d8014d9b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c800 tx=0x7f7d7400ccc0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 shutdown_connections 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f7d6c078bc0 0x7f7d6c07b0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c7af0 0x7f7d80150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d800c2e60 0x7f7d8014d9b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 --2- 172.21.15.88:0/3888029778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d8006de70 0x7f7d801f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 >> 172.21.15.88:0/3888029778 conn(0x7f7d800720b0 msgr2=0x7f7d80076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 shutdown_connections 2024-09-21T02:07:45.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.726+0000 7f7d85607640 1 -- 172.21.15.88:0/3888029778 wait complete. 2024-09-21T02:07:45.740 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:07:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 -- 172.21.15.88:0/2290229948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c9b10 msgr2=0x7f82040c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 --2- 172.21.15.88:0/2290229948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c9b10 0x7f82040c9f20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f81f8005fd0 tx=0x7f81f802a5d0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 -- 172.21.15.88:0/2290229948 shutdown_connections 2024-09-21T02:07:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 --2- 172.21.15.88:0/2290229948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c9b10 0x7f82040c9f20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 --2- 172.21.15.88:0/2290229948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82040c4130 0x7f8204153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 -- 172.21.15.88:0/2290229948 >> 172.21.15.88:0/2290229948 conn(0x7f8204068d00 msgr2=0x7f8204069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 -- 172.21.15.88:0/2290229948 shutdown_connections 2024-09-21T02:07:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.856+0000 7f820a5ba640 1 -- 172.21.15.88:0/2290229948 wait complete. 2024-09-21T02:07:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.857+0000 7f820a5ba640 1 Processor -- start 2024-09-21T02:07:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.858+0000 7f820a5ba640 1 -- start start 2024-09-21T02:07:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f820a5ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c4130 0x7f82040ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f820a5ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82040c9b10 0x7f820414b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f820a5ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82040cbba0 0x7f820414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f820a5ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f820415a190 con 0x7f82040c4130 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f820a5ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82040cb620 con 0x7f82040cbba0 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f820a5ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f820415a310 con 0x7f82040c9b10 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c4130 0x7f82040ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c4130 0x7f82040ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48326/0 (socket says 172.21.15.88:48326) 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 -- 172.21.15.88:0/3380432099 learned_addr learned my addr 172.21.15.88:0/3380432099 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8208b57640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82040cbba0 0x7f820414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f82037fe640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82040c9b10 0x7f820414b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 -- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82040c9b10 msgr2=0x7f820414b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82040c9b10 0x7f820414b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 -- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82040cbba0 msgr2=0x7f820414bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82040cbba0 0x7f820414bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8203fff640 1 -- 172.21.15.88:0/3380432099 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82041e4c70 con 0x7f82040c4130 2024-09-21T02:07:45.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f82037fe640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82040c9b10 0x7f820414b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:45.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.859+0000 7f8208b57640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82040cbba0 0x7f820414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:07:45.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.860+0000 7f8203fff640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c4130 0x7f82040ccf80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f81f400d700 tx=0x7f81f400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:45.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.860+0000 7f8200ff9640 1 -- 172.21.15.88:0/3380432099 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4017070 con 0x7f82040c4130 2024-09-21T02:07:45.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.860+0000 7f8200ff9640 1 -- 172.21.15.88:0/3380432099 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f81f4005330 con 0x7f82040c4130 2024-09-21T02:07:45.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.860+0000 7f8200ff9640 1 -- 172.21.15.88:0/3380432099 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4009820 con 0x7f82040c4130 2024-09-21T02:07:45.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.860+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82041e51c0 con 0x7f82040c4130 2024-09-21T02:07:45.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.860+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82041508c0 con 0x7f82040c4130 2024-09-21T02:07:45.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.861+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8204153720 con 0x7f82040c4130 2024-09-21T02:07:45.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.862+0000 7f8200ff9640 1 -- 172.21.15.88:0/3380432099 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81f40099c0 con 0x7f82040c4130 2024-09-21T02:07:45.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.862+0000 7f8200ff9640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f81f0078b90 0x7f81f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:45.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.862+0000 7f8200ff9640 1 -- 172.21.15.88:0/3380432099 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6083+0+0 (secure 0 0 0) 0x7f81f409af40 con 0x7f82040c4130 2024-09-21T02:07:45.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.862+0000 7f82037fe640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f81f0078b90 0x7f81f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:45.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.863+0000 7f82037fe640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f81f0078b90 0x7f81f007b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f81f8033000 tx=0x7f81f80046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:45.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:45.867+0000 7f8200ff9640 1 -- 172.21.15.88:0/3380432099 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f81f4063290 con 0x7f82040c4130 2024-09-21T02:07:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.045+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f82040b2360 con 0x7f82040c4130 2024-09-21T02:07:46.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.726+0000 7f8200ff9640 1 -- 172.21.15.88:0/3380432099 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f81f4062b90 con 0x7f82040c4130 2024-09-21T02:07:46.727 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:07:46.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f81f0078b90 msgr2=0x7f81f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f81f0078b90 0x7f81f007b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f81f8033000 tx=0x7f81f80046c0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c4130 msgr2=0x7f82040ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c4130 0x7f82040ccf80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f81f400d700 tx=0x7f81f400dbc0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 shutdown_connections 2024-09-21T02:07:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f81f0078b90 0x7f81f007b0a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82040cbba0 0x7f820414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82040c9b10 0x7f820414b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 --2- 172.21.15.88:0/3380432099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82040c4130 0x7f82040ccf80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 >> 172.21.15.88:0/3380432099 conn(0x7f8204068d00 msgr2=0x7f8204156610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 shutdown_connections 2024-09-21T02:07:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.729+0000 7f820a5ba640 1 -- 172.21.15.88:0/3380432099 wait complete. 2024-09-21T02:07:46.745 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:07:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.863+0000 7f83029f1640 1 -- 172.21.15.88:0/2851601163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc075700 msgr2=0x7f82fc072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.863+0000 7f83029f1640 1 --2- 172.21.15.88:0/2851601163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc075700 0x7f82fc072e40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009830 tx=0x7f82ec02a640 comp rx=0 tx=0).stop 2024-09-21T02:07:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.863+0000 7f83029f1640 1 -- 172.21.15.88:0/2851601163 shutdown_connections 2024-09-21T02:07:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.863+0000 7f83029f1640 1 --2- 172.21.15.88:0/2851601163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc075700 0x7f82fc072e40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.863+0000 7f83029f1640 1 -- 172.21.15.88:0/2851601163 >> 172.21.15.88:0/2851601163 conn(0x7f82fc06d8f0 msgr2=0x7f82fc0ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.863+0000 7f83029f1640 1 -- 172.21.15.88:0/2851601163 shutdown_connections 2024-09-21T02:07:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.863+0000 7f83029f1640 1 -- 172.21.15.88:0/2851601163 wait complete. 2024-09-21T02:07:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.865+0000 7f83029f1640 1 Processor -- start 2024-09-21T02:07:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.865+0000 7f83029f1640 1 -- start start 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.866+0000 7f83029f1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc073850 0x7f82fc157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.866+0000 7f83029f1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82fc075700 0x7f82fc1e4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.866+0000 7f83029f1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82fc153970 0x7f82fc147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.866+0000 7f83029f1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc159e70 con 0x7f82fc073850 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.866+0000 7f83029f1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc0c9380 con 0x7f82fc153970 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.866+0000 7f83029f1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc0c7f10 con 0x7f82fc075700 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc073850 0x7f82fc157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc073850 0x7f82fc157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48346/0 (socket says 172.21.15.88:48346) 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 -- 172.21.15.88:0/3211570142 learned_addr learned my addr 172.21.15.88:0/3211570142 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f8300f8e640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82fc153970 0x7f82fc147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fb7fe640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82fc075700 0x7f82fc1e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 -- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82fc075700 msgr2=0x7f82fc1e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82fc075700 0x7f82fc1e4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 -- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82fc153970 msgr2=0x7f82fc147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82fc153970 0x7f82fc147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 -- 172.21.15.88:0/3211570142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc157e80 con 0x7f82fc073850 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82fbfff640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc073850 0x7f82fc157940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009920 tx=0x7f82ec0055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82f8ff9640 1 -- 172.21.15.88:0/3211570142 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec031ad0 con 0x7f82fc073850 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82f8ff9640 1 -- 172.21.15.88:0/3211570142 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f82ec031d80 con 0x7f82fc073850 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f82f8ff9640 1 -- 172.21.15.88:0/3211570142 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec033330 con 0x7f82fc073850 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc1e3990 con 0x7f82fc073850 2024-09-21T02:07:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.867+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc154510 con 0x7f82fc073850 2024-09-21T02:07:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.868+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82fc0c8220 con 0x7f82fc073850 2024-09-21T02:07:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.869+0000 7f82f8ff9640 1 -- 172.21.15.88:0/3211570142 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f82ec0334d0 con 0x7f82fc073850 2024-09-21T02:07:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.869+0000 7f82f8ff9640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f82e8078b90 0x7f82e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:07:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.870+0000 7f82f8ff9640 1 -- 172.21.15.88:0/3211570142 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5645+0+0 (secure 0 0 0) 0x7f82ec0bbf00 con 0x7f82fc073850 2024-09-21T02:07:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.870+0000 7f82fb7fe640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f82e8078b90 0x7f82e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:07:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.870+0000 7f82fb7fe640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f82e8078b90 0x7f82e807b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f82fc1e4210 tx=0x7f82f0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:07:46.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:46.874+0000 7f82f8ff9640 1 -- 172.21.15.88:0/3211570142 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f82ec03f030 con 0x7f82fc073850 2024-09-21T02:07:47.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.059+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f82fc1e56d0 con 0x7f82fc073850 2024-09-21T02:07:47.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.731+0000 7f82f8ff9640 1 -- 172.21.15.88:0/3211570142 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f82ec083dc0 con 0x7f82fc073850 2024-09-21T02:07:47.732 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f82e8078b90 msgr2=0x7f82e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f82e8078b90 0x7f82e807b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f82fc1e4210 tx=0x7f82f0009040 comp rx=0 tx=0).stop 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc073850 msgr2=0x7f82fc157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc073850 0x7f82fc157940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009920 tx=0x7f82ec0055e0 comp rx=0 tx=0).stop 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 shutdown_connections 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:6800/4016634266,v1:172.21.15.195:6801/4016634266] conn(0x7f82e8078b90 0x7f82e807b0a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82fc153970 0x7f82fc147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82fc075700 0x7f82fc1e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 --2- 172.21.15.88:0/3211570142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82fc073850 0x7f82fc157940 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:07:47.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 >> 172.21.15.88:0/3211570142 conn(0x7f82fc06d8f0 msgr2=0x7f82fc149bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:07:47.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 shutdown_connections 2024-09-21T02:07:47.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T02:07:47.734+0000 7f83029f1640 1 -- 172.21.15.88:0/3211570142 wait complete. 2024-09-21T02:07:47.747 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T02:07:47.748 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:07:47.748 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:47.761 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:07:47.761 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:07:47.761 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:47.812 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:07:47.812 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-21T02:07:47.812 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:47.863 INFO:tasks.ceph.mds.e:Stopped 2024-09-21T02:07:47.863 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:07:47.863 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:47.874 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:07:47.875 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:07:47.875 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:47.928 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:07:47.929 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-21T02:07:47.929 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:47.982 INFO:tasks.ceph.mds.f:Stopped 2024-09-21T02:07:47.982 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T02:07:47.983 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T02:07:47.983 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.012 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T02:07:48.012 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T02:07:48.012 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.082 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T02:07:48.082 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T02:07:48.083 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.149 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T02:07:48.150 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T02:07:48.150 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.225 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T02:07:48.225 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T02:07:48.225 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.257 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T02:07:48.258 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T02:07:48.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.330 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T02:07:48.330 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T02:07:48.330 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.400 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T02:07:48.400 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T02:07:48.400 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.477 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T02:07:48.503 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T02:07:48.503 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T02:07:48.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.530 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T02:07:48.530 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T02:07:48.530 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.563 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T02:07:48.563 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T02:07:48.563 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T02:07:48.563 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.582 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T02:07:48.582 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T02:07:48.583 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.617 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T02:07:48.617 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T02:07:48.617 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:07:48.671 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T02:07:48.672 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T02:07:48.672 DEBUG:teuthology.orchestra.run.smithi088:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-09-21T02:07:48.717 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:07:48.717 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T02:07:48.881 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:07:48.881 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T02:07:48.992 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:07:48.992 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T02:07:49.041 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:07:49.041 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T02:07:49.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:07:49.135 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T02:07:49.329 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:07:49.330 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T02:07:49.414 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:07:49.414 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T02:07:49.489 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:07:49.489 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T02:07:49.547 INFO:tasks.ceph:Archiving mon data... 2024-09-21T02:07:49.549 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/data/mon.a.tgz 2024-09-21T02:07:49.549 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-09-21T02:07:49.565 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.XDooYWHnRw 2024-09-21T02:07:49.566 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.XDooYWHnRw 2024-09-21T02:07:49.704 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.XDooYWHnRw 2024-09-21T02:07:49.786 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.XDooYWHnRw is 329KB 2024-09-21T02:07:49.852 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.XDooYWHnRw 2024-09-21T02:07:49.868 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/data/mon.b.tgz 2024-09-21T02:07:49.868 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-21T02:07:49.886 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.IlirZAJCdX 2024-09-21T02:07:49.886 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.IlirZAJCdX 2024-09-21T02:07:50.045 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.IlirZAJCdX 2024-09-21T02:07:50.129 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.IlirZAJCdX is 437KB 2024-09-21T02:07:50.197 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.IlirZAJCdX 2024-09-21T02:07:50.212 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/data/mon.c.tgz 2024-09-21T02:07:50.212 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-21T02:07:50.267 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.x1R9rLhfe0 2024-09-21T02:07:50.267 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.x1R9rLhfe0 2024-09-21T02:07:50.393 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.x1R9rLhfe0 2024-09-21T02:07:50.475 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.x1R9rLhfe0 is 361KB 2024-09-21T02:07:50.570 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.x1R9rLhfe0 2024-09-21T02:07:50.585 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T02:07:50.586 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T02:07:50.590 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T02:07:50.670 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T02:07:50.705 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T02:07:50.705 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T02:07:50.707 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/remote/smithi088/crash 2024-09-21T02:07:50.708 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T02:07:50.737 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/remote/smithi195/crash 2024-09-21T02:07:50.738 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T02:07:50.769 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T02:07:50.770 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:07:50.782 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:07:50.867 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T02:07:50.893 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T02:07:50.917 INFO:tasks.ceph:Compressing logs... 2024-09-21T02:07:50.918 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:07:50.936 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49250.log 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.49250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44090.log 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49250.log.gz 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42911.log 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50048.log 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44090.log: /var/log/ceph/ceph-client.admin.42911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46858.log 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44090.log.gz 2024-09-21T02:07:50.960 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42911.log.gz 2024-09-21T02:07:50.961 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47175.log 2024-09-21T02:07:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50048.log: /var/log/ceph/ceph-client.admin.46858.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2024-09-21T02:07:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49586.log 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47175.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46858.log.gz 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43746.log 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47175.log.gz 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49586.log.gz 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43746.log.gz 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49712.log 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47890.log 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48481.log 2024-09-21T02:07:50.984 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49712.log.gz 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9%/var/log/ceph/ceph-client.admin.48481.log: -- replaced with /var/log/ceph/ceph-client.admin.47890.log.gz 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48271.log 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48481.log.gz 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44096.log 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43300.log 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48271.log.gz 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46923.log 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44096.log.gz 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43300.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43300.log.gz 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46183.log 2024-09-21T02:07:50.985 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46923.log.gz 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49376.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44755.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46183.log.gz 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36526.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49376.log.gz 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50274.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44755.log: /var/log/ceph/ceph-client.admin.36526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50515.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44755.log.gz 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36526.log.gz 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50274.log.gz 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46099.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50515.log.gz 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45091.log 2024-09-21T02:07:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46552.log: /var/log/ceph/ceph-client.admin.46099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49796.log 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46099.log.gz 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50473.log 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45091.log.gz 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43788.log 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49796.log: /var/log/ceph/ceph-client.admin.50473.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49796.log.gz 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50317.log 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50473.log.gz 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36413.log 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43788.log: /var/log/ceph/ceph-client.admin.50317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47567.log 2024-09-21T02:07:50.987 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43788.log.gz 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50317.log.gz 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36413.log.gz 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44758.log 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36792.log 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47567.log.gz 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44758.log.gz 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43696.log 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36792.log: /var/log/ceph/ceph-client.admin.46636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46732.log 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36792.log.gz 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37133.log 2024-09-21T02:07:50.988 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43696.log.gz 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45346.log 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46732.log.gz 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45301.log 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43612.log 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45346.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37133.log.gz 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45301.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45346.log.gz 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47343.log 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45301.log.gz 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46510.log 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43612.log: /var/log/ceph/ceph-client.admin.47343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43612.log.gz 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47343.log.gz 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46309.log 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46510.log.gzgzip 2024-09-21T02:07:50.989 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43256.log 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47217.log: /var/log/ceph/ceph-client.admin.46309.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43914.log 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46309.log.gz 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43256.log: /var/log/ceph/ceph-client.admin.43914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43026.log 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43256.log.gz 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43914.log.gz 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T02:07:50.990 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43026.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-09-21T02:07:50.991 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47609.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43026.log.gz 2024-09-21T02:07:50.991 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46141.log 2024-09-21T02:07:50.991 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-09-21T02:07:50.991 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2024-09-21T02:07:50.991 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.audit.log: gzip/var/log/ceph/ceph-client.admin.46141.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49838.log 2024-09-21T02:07:50.991 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48229.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46141.log.gz 2024-09-21T02:07:50.991 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44087.log 2024-09-21T02:07:50.992 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2024-09-21T02:07:50.992 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44765.log 2024-09-21T02:07:50.992 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49838.log.gz 2024-09-21T02:07:50.992 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T02:07:50.992 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2024-09-21T02:07:50.992 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44087.log: /var/log/ceph/ceph-client.admin.44765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43654.log 2024-09-21T02:07:50.993 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44087.log.gz 2024-09-21T02:07:50.993 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44765.log.gz 2024-09-21T02:07:50.993 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43170.log 2024-09-21T02:07:50.993 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2024-09-21T02:07:50.993 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30369.log 2024-09-21T02:07:50.993 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43654.log: /var/log/ceph/ceph-client.admin.43170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46965.log 2024-09-21T02:07:50.993 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43654.log.gz 2024-09-21T02:07:50.994 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43170.log.gz 2024-09-21T02:07:50.994 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.tmp-client.admin.30369.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30369.log.gz 2024-09-21T02:07:50.994 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49036.log 2024-09-21T02:07:50.994 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2024-09-21T02:07:50.994 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46965.log.gz 2024-09-21T02:07:50.994 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43830.log 2024-09-21T02:07:50.994 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49036.log.gz 2024-09-21T02:07:50.995 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2024-09-21T02:07:50.995 INFO:teuthology.orchestra.run.smithi088.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2024-09-21T02:07:50.995 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37134.log 2024-09-21T02:07:50.995 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43830.log: gzip/var/log/ceph/ceph-client.admin.44429.log: -5 --verbose -- /var/log/ceph/ceph-client.0.45961.log 2024-09-21T02:07:50.996 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43830.log.gz 2024-09-21T02:07:50.996 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gz 2024-09-21T02:07:50.996 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-09-21T02:07:50.996 INFO:teuthology.orchestra.run.smithi088.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.37134.log.gz 2024-09-21T02:07:50.996 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44628.log 2024-09-21T02:07:50.996 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.45961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45961.log.gz 2024-09-21T02:07:50.996 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49502.log 2024-09-21T02:07:50.997 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz 2024-09-21T02:07:50.997 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48313.log 2024-09-21T02:07:50.997 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44628.log.gz 2024-09-21T02:07:50.997 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46816.log 2024-09-21T02:07:50.997 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49502.log: /var/log/ceph/ceph-client.admin.48313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-09-21T02:07:50.997 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49502.log.gz 2024-09-21T02:07:50.997 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48313.log.gz 2024-09-21T02:07:50.998 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48952.log 2024-09-21T02:07:50.998 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T02:07:50.998 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46816.log.gz/var/log/ceph/ceph-client.admin.44766.log: 2024-09-21T02:07:50.998 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48952.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-09-21T02:07:50.998 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44428.log 2024-09-21T02:07:50.998 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48952.log.gz 2024-09-21T02:07:50.998 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44424.log 2024-09-21T02:07:50.999 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.44428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43212.log 2024-09-21T02:07:50.999 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz 2024-09-21T02:07:50.999 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48627.log 2024-09-21T02:07:50.999 INFO:teuthology.orchestra.run.smithi088.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.44424.log.gz 2024-09-21T02:07:50.999 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47848.log 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43212.log.gz 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44427.log 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48627.log: /var/log/ceph/ceph-client.admin.47848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45049.log 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48627.log.gz 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47848.log.gz 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46690.log 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr:gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44427.log.gz -- 2024-09-21T02:07:51.000 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45049.log: /var/log/ceph/ceph-client.admin.48826.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37084.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37164.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37084.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37084.log.gz 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37164.log.gz 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37244.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T02:07:51.015 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37364.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37324.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37364.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.37364.log.gz 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37324.log: 93.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37324.log.gz 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37204.log 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mgr.x.log: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T02:07:51.016 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T02:07:51.017 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37204.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37204.log.gz 2024-09-21T02:07:51.017 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37284.log 2024-09-21T02:07:51.017 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37124.log 2024-09-21T02:07:51.017 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37284.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37284.log.gz 2024-09-21T02:07:51.028 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-09-21T02:07:51.029 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.37124.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37124.log.gz 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45049.log.gz 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46690.log.gz 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44768.log 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48826.log: /var/log/ceph/ceph-client.admin.44092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44767.log 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48826.log.gz 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44768.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44768.log.gz 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44767.log.gz 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T02:07:51.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46225.log: /var/log/ceph/ceph-client.admin.44094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42767.log 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43528.log 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44086.log 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42767.log: /var/log/ceph/ceph-client.admin.43528.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42767.log.gz 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43528.log.gz 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44086.log.gz 2024-09-21T02:07:51.058 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49922.log 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37596.log: /var/log/ceph/ceph-client.admin.50232.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49628.log 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45966.log 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49922.log.gz 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49418.log 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49628.log.gz 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.45966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46467.log 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45966.log.gz 2024-09-21T02:07:51.059 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49880.log 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49418.log: /var/log/ceph/ceph-client.admin.46467.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49418.log.gz 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49544.log 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46467.log.gz 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37797.log 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49880.log.gz 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49754.log 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49544.log: /var/log/ceph/ceph-client.admin.37797.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49544.log.gz 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37797.log.gz 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44411.log 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49754.log: /var/log/ceph/ceph-client.admin.43486.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49110.log 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49754.log.gz 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-09-21T02:07:51.060 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47259.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49110.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44411.log.gz 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50150.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49110.log.gz 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47259.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47259.log.gz 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr: -5/var/log/ceph/ceph-client.admin.50150.log: --verbose -- /var/log/ceph/ceph-client.admin.44088.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.2.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50150.log.gz 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36934.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36582.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1%/var/log/ceph/ceph-client.admin.36934.log: -- replaced with /var/log/ceph/ceph-client.admin.44088.log.gz 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45520.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36934.log.gz 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36582.log: /var/log/ceph/ceph-client.admin.45520.log: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.36582.log.gz 2024-09-21T02:07:51.061 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37008.log 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48585.log 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45520.log.gz 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48784.log: /var/log/ceph/ceph-client.admin.37008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37012.log 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48585.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37008.log.gz 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48585.log.gz 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36631.log 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37012.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48439.log 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.37012.log.gz 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36631.log: 79.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37023.log 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36631.log.gz 2024-09-21T02:07:51.062 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48439.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48439.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43570.log 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37887.log 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37023.log: /var/log/ceph/ceph-client.admin.43570.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37023.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43570.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44964.log 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47049.log 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44964.log: /var/log/ceph/ceph-client.admin.37887.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.37887.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44964.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47049.log: /var/log/ceph/ceph.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.47049.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47806.log 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37066.log: 93.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-09-21T02:07:51.063 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 85.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44432.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37092.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47806.log.gz 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46267.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44432.log.gz 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37099.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.37092.log.gz 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45622.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46267.log: /var/log/ceph/ceph-client.admin.37099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47658.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46267.log.gz 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45622.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37099.log.gz 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45622.log.gz 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50006.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.47658.log: --verbose -- /var/log/ceph/ceph-client.admin.48910.log 2024-09-21T02:07:51.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50006.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47658.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50006.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.48910.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36728.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.48910.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36692.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36728.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37007.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36692.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45133.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.36692.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37007.log.gz 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48724.log 2024-09-21T02:07:51.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44430.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45133.log.gz 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44091.log 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37106.log 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48724.log: /var/log/ceph/ceph-client.admin.44091.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48724.log.gz 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44091.log.gz 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47722.log 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43872.log 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr: 85.7% 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37106.log.gz/var/log/ceph/ceph-client.admin.47722.log: 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49670.log 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47722.log.gz 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50411.log 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43872.log.gz 2024-09-21T02:07:51.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47301.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49670.log.gz 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48868.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50411.log.gz 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49334.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47301.log.gz 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37135.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49334.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48868.log.gz 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49460.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49334.log.gz 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37135.log: gzip -5 --verbose -- 84.9% -- replaced with /var/log/ceph/ceph-client.admin.37135.log.gz 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.43402.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49460.log: gzip -5 --verbose 86.9% -- -- replaced with /var/log/ceph/ceph-client.admin.49460.log.gz 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.48397.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50359.log 2024-09-21T02:07:51.067 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43402.log.gz 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36498.log 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.48397.log.gz 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46774.log 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50359.log.gz 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36498.log: 74.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.36498.log.gz 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50190.log 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36997.log 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4%/var/log/ceph/ceph-client.admin.50190.log: -- replaced with /var/log/ceph/ceph-client.admin.46774.log.gz 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44764.log 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50190.log.gz 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45388.log 2024-09-21T02:07:51.068 INFO:teuthology.orchestra.run.smithi088.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36997.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44764.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44764.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47091.log 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43360.log 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45388.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47091.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43128.log 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43360.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37556.log: gzip -5 93.7% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36470.log 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43128.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43128.log.gz 2024-09-21T02:07:51.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46594.log 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr: 75.7% -- replaced with /var/log/ceph/ceph-client.admin.36470.log.gz 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47386.log 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44431.log 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0%/var/log/ceph/ceph-client.admin.47386.log: -- replaced with /var/log/ceph/ceph-client.admin.46594.log.gz 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48355.log 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47386.log.gz 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43444.log 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44431.log: /var/log/ceph/ceph-client.admin.48355.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43444.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44431.log.gz 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48355.log.gz 2024-09-21T02:07:51.070 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49964.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43444.log.gz 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37124.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36994.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49964.log: /var/log/ceph/ceph-client.admin.37124.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49964.log.gz 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.37124.log.gz 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36903.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36994.log: 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.36994.log.gz 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45175.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47764.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36903.log.gz 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45006.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45175.log.gz 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36554.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47764.log: gzip/var/log/ceph/ceph-client.admin.45006.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49292.log 2024-09-21T02:07:51.071 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2%/var/log/ceph/ceph-client.admin.36554.log: -- replaced with /var/log/ceph/ceph-client.admin.45006.log.gz 2024-09-21T02:07:51.072 INFO:teuthology.orchestra.run.smithi088.stderr: 75.9% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47764.log.gz 2024-09-21T02:07:51.072 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36554.log.gz 2024-09-21T02:07:51.072 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49292.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49292.log.gz 2024-09-21T02:07:51.072 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mds.f.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T02:07:51.082 INFO:teuthology.orchestra.run.smithi195.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-09-21T02:07:51.092 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T02:07:51.101 INFO:teuthology.orchestra.run.smithi195.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T02:07:51.282 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T02:07:51.333 INFO:teuthology.orchestra.run.smithi088.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T02:07:51.378 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T02:07:51.528 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T02:07:51.579 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T02:07:51.638 INFO:teuthology.orchestra.run.smithi195.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T02:07:51.798 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T02:07:52.053 INFO:teuthology.orchestra.run.smithi195.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T02:07:52.090 INFO:teuthology.orchestra.run.smithi088.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T02:07:52.092 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T02:07:52.092 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m1.144s 2024-09-21T02:07:52.092 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m2.701s 2024-09-21T02:07:52.092 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m0.223s 2024-09-21T02:07:52.278 INFO:teuthology.orchestra.run.smithi195.stderr: 95.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T02:07:52.280 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-09-21T02:07:52.280 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m1.305s 2024-09-21T02:07:52.280 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m4.567s 2024-09-21T02:07:52.280 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m0.180s 2024-09-21T02:07:52.280 INFO:tasks.ceph:Archiving logs... 2024-09-21T02:07:52.281 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/remote/smithi088/log 2024-09-21T02:07:52.281 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:07:52.797 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/remote/smithi195/log 2024-09-21T02:07:52.798 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:07:53.347 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T02:07:53.356 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T02:07:53.356 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:07:53.359 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:07:53.441 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T02:07:53.452 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T02:07:53.453 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:07:53.456 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:07:53.470 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:07:53.470 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:07:53.476 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:07:53.476 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-09-21T02:07:53.476 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 19 +69us[ +70us] +/- 65ms 2024-09-21T02:07:53.476 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -719us[ -718us] +/- 58ms 2024-09-21T02:07:53.476 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +819us[ +820us] +/- 69ms 2024-09-21T02:07:53.476 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:07:53.477 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:07:53.477 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-09-21T02:07:53.477 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 19 +41us[ +37us] +/- 65ms 2024-09-21T02:07:53.477 INFO:teuthology.orchestra.run.smithi088.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -730us[ -734us] +/- 58ms 2024-09-21T02:07:53.477 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +818us[ +814us] +/- 69ms 2024-09-21T02:07:53.477 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:07:53.477 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T02:07:53.487 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T02:07:53.488 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T02:07:53.497 DEBUG:teuthology.orchestra.run.smithi088:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:07:53.588 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T02:07:53.649 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T02:07:53.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:07:53.686 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T02:07:53.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:07:53.715 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T02:07:53.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T02:07:53.756 INFO:teuthology.task.internal:Duration was 650.177310 seconds 2024-09-21T02:07:53.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T02:07:53.768 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T02:07:53.768 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:07:53.772 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:07:53.810 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:07:53.811 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:07:54.193 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T02:07:54.193 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T02:07:54.193 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:07:54.257 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-09-21T02:07:54.257 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:07:54.289 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T02:07:54.289 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:07:54.300 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:07:54.358 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T02:07:54.358 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:07:54.372 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:07:54.498 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T02:07:54.507 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T02:07:54.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:07:54.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:07:54.557 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T02:07:54.568 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:07:54.575 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:07:54.596 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-09-21T02:07:54.621 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-09-21T02:07:54.644 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:07:54.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:07:54.681 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:07:54.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:07:54.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T02:07:54.709 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T02:07:54.710 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/remote/smithi088 2024-09-21T02:07:54.710 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:07:54.755 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913405/remote/smithi195 2024-09-21T02:07:54.755 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:07:54.788 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T02:07:54.788 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:07:54.792 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:07:54.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T02:07:54.850 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T02:07:54.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T02:07:54.860 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T02:07:54.860 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:07:54.862 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:07:54.875 INFO:teuthology.orchestra.run.smithi088.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:07 /home/ubuntu/cephtest 2024-09-21T02:07:54.894 INFO:teuthology.orchestra.run.smithi195.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:07 /home/ubuntu/cephtest 2024-09-21T02:07:54.895 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T02:07:54.912 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T02:07:55.045 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} duration: 650.1773104667664 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T02:07:55.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:07:55.129 INFO:teuthology.run:pass